[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:12 dist-test-slave-8w9v krb5kdc[15413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:12 dist-test-slave-8w9v krb5kdc[15413](info): set up 2 sockets
Aug 09 15:13:12 dist-test-slave-8w9v krb5kdc[15413](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 15:13:13.342063 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:13 dist-test-slave-8w9v krb5kdc[15413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:13.528688 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:38703
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:38703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:13.862162 15428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:13.862519 15428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:13.862715 15428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:13.869911 15428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:13.870069 15428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:13.870190 15428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:13.870344 15428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:13.870445 15428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:13.879848 15428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:38703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:38703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:13.881162 15428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:13.882525 15428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:13.895831 15438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:13.895860 15435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:13.895846 15436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:13.897753 15428 server_base.cc:1034] running on GCE node
I20260809 15:13:13.898557 15428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:13.899936 15428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:13.901181 15428 hybrid_clock.cc:648] HybridClock initialized: now 1786288393901176 us; error 92 us; skew 500 ppm
I20260809 15:13:13.901571 15428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:13 dist-test-slave-8w9v krb5kdc[15413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288393, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:13.904816 15428 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:13.906242 15428 webserver.cc:460] Webserver started at http://127.15.11.190:45535/ using document root <none> and password file <none>
I20260809 15:13:13.907002 15428 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:13.907172 15428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:13.907483 15428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:13.910447 15428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "2388e6da31d3472389b11683432c81c9"
format_stamp: "Formatted at 2026-08-09 15:13:13 on dist-test-slave-8w9v"
server_key: "21aca5501aa2bf37971a00c0029de4f3"
server_key_iv: "db2a732091e88b478a8f24908f17797c"
server_key_version: "encryptionkey@0"
I20260809 15:13:13.911243 15428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "2388e6da31d3472389b11683432c81c9"
format_stamp: "Formatted at 2026-08-09 15:13:13 on dist-test-slave-8w9v"
server_key: "21aca5501aa2bf37971a00c0029de4f3"
server_key_iv: "db2a732091e88b478a8f24908f17797c"
server_key_version: "encryptionkey@0"
I20260809 15:13:13.916424 15428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 15:13:13.919750 15445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:13.921134 15428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:13.921315 15428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "2388e6da31d3472389b11683432c81c9"
format_stamp: "Formatted at 2026-08-09 15:13:13 on dist-test-slave-8w9v"
server_key: "21aca5501aa2bf37971a00c0029de4f3"
server_key_iv: "db2a732091e88b478a8f24908f17797c"
server_key_version: "encryptionkey@0"
I20260809 15:13:13.921509 15428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:13.935043 15428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:13.938791 15428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:13.939029 15428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:13.957881 15428 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:38703
I20260809 15:13:13.957903 15501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:38703 every 8 connection(s)
I20260809 15:13:13.959206 15428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:13.963042 15502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:13.965764 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 15428
I20260809 15:13:13.966076 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:13.967998 15406 external_mini_cluster.cc:1428] Setting key 0b868f7a3088951dbd302aea28b7ced9
I20260809 15:13:13.973881 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9: Bootstrap starting.
I20260809 15:13:13.977044 15502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:13.978119 15502 log.cc:824] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:13.980983 15502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9: No bootstrap required, opened a new log
I20260809 15:13:13.991074 15502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2388e6da31d3472389b11683432c81c9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38703 } }
I20260809 15:13:13.991566 15502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:13.991693 15502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2388e6da31d3472389b11683432c81c9, State: Initialized, Role: FOLLOWER
I20260809 15:13:13.992409 15502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2388e6da31d3472389b11683432c81c9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38703 } }
I20260809 15:13:13.992671 15502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:13.992808 15502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:13.992976 15502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:13.995270 15502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2388e6da31d3472389b11683432c81c9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38703 } }
I20260809 15:13:13.995765 15502 leader_election.cc:302] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2388e6da31d3472389b11683432c81c9; no voters:
I20260809 15:13:13.996481 15502 leader_election.cc:288] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:13.996592 15508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [term 1 FOLLOWER]: Leader election won for term 1
Aug 09 15:13:13 dist-test-slave-8w9v krb5kdc[15413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:13.998132 15508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [term 1 LEADER]: Becoming Leader. State: Replica: 2388e6da31d3472389b11683432c81c9, State: Running, Role: LEADER
I20260809 15:13:13.998908 15502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:13.998739 15508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2388e6da31d3472389b11683432c81c9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38703 } }
I20260809 15:13:14.009929 15504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:13.972038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.15.11.190:38703)
0809 15:13:13.973044 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:13.973059 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:13.974433 (+ 1374us) server_negotiation.cc:407] Connection header received
0809 15:13:13.983687 (+ 9254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:13.983713 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:13.984218 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:13.984647 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:13.987251 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:13.989319 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:13.990933 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:13.992021 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:13.992234 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:13.998746 (+ 6512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:13.998760 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:13.998780 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:13.998838 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:14.004296 (+ 5458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:14.005495 (+ 1199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:14.005506 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:14.005518 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:14.005683 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:14.006192 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:14.006199 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:14.006202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:14.006778 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:14.007130 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:14.008000 (+ 870us) server_negotiation.cc:299] Negotiation successful
0809 15:13:14.008467 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":467,"threads_started":1}
I20260809 15:13:14.009617 15510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2388e6da31d3472389b11683432c81c9. Latest consensus state: current_term: 1 leader_uuid: "2388e6da31d3472389b11683432c81c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2388e6da31d3472389b11683432c81c9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38703 } } }
I20260809 15:13:14.010121 15510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:14.013725 15509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2388e6da31d3472389b11683432c81c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2388e6da31d3472389b11683432c81c9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38703 } } }
I20260809 15:13:14.014091 15509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9 [sys.catalog]: This master's current role is: LEADER
W20260809 15:13:14.022428 15526 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:13:14.022563 15526 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:13:14.022711 15525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:14.024930 15525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:14.033396 15525 catalog_manager.cc:1347] Generated new cluster ID: 4224a61ebc7b424d94bac6f05dbfb7eb
I20260809 15:13:14.033499 15525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:14.063953 15525 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:14.065423 15525 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:14.072412 15525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2388e6da31d3472389b11683432c81c9: Generated new TSK 0
I20260809 15:13:14.073256 15525 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:13:14.124415 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38703
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:14.459363 15532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:14.459623 15532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:14.459754 15532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:14.466915 15532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:14.467007 15532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:14.467190 15532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:14.476435 15532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:14.477634 15532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:14.478876 15532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:14.491775 15540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:14.491750 15537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:14.491724 15538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:14.493088 15532 server_base.cc:1034] running on GCE node
I20260809 15:13:14.493777 15532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:14.494807 15532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:14.496055 15532 hybrid_clock.cc:648] HybridClock initialized: now 1786288394495994 us; error 44 us; skew 500 ppm
I20260809 15:13:14.496402 15532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:14 dist-test-slave-8w9v krb5kdc[15413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288394, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:14.499747 15532 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:13:14.501436 15532 webserver.cc:460] Webserver started at http://127.15.11.129:46083/ using document root <none> and password file <none>
I20260809 15:13:14.502140 15532 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:14.502326 15532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:14.502679 15532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:14.505797 15532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "aad6ce9875e1449e9fb0628174bfe158"
format_stamp: "Formatted at 2026-08-09 15:13:14 on dist-test-slave-8w9v"
server_key: "e2d529133d2313aa160aaf57cec46cfc"
server_key_iv: "02d67cee6c349da661e96ca94b998721"
server_key_version: "encryptionkey@0"
I20260809 15:13:14.506584 15532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "aad6ce9875e1449e9fb0628174bfe158"
format_stamp: "Formatted at 2026-08-09 15:13:14 on dist-test-slave-8w9v"
server_key: "e2d529133d2313aa160aaf57cec46cfc"
server_key_iv: "02d67cee6c349da661e96ca94b998721"
server_key_version: "encryptionkey@0"
I20260809 15:13:14.512001 15532 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:13:14.515594 15547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:14.516990 15532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:14.517184 15532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "aad6ce9875e1449e9fb0628174bfe158"
format_stamp: "Formatted at 2026-08-09 15:13:14 on dist-test-slave-8w9v"
server_key: "e2d529133d2313aa160aaf57cec46cfc"
server_key_iv: "02d67cee6c349da661e96ca94b998721"
server_key_version: "encryptionkey@0"
I20260809 15:13:14.517393 15532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:14.540653 15532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:14.544896 15532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:14.545230 15532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:14.546473 15532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:14.547883 15532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:14.547997 15532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:14.548195 15532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:14.548273 15532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 15:13:14.583328 15532 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:34463
I20260809 15:13:14.583437 15668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:34463 every 8 connection(s)
I20260809 15:13:14.584805 15532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:13:14.593923 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 15532
I20260809 15:13:14.594187 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:13:14.594996 15406 external_mini_cluster.cc:1428] Setting key c8ff0339170939803c20857de4ee46d6
Aug 09 15:13:14 dist-test-slave-8w9v krb5kdc[15413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288394, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:14.607211 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:14.587263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:55013 (local address 127.15.11.190:38703)
0809 15:13:14.587718 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:14.587722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:14.588612 (+ 890us) server_negotiation.cc:407] Connection header received
0809 15:13:14.590443 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:14.590451 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:14.590539 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:14.590658 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:14.592124 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:14.592840 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:14.595021 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:14.595502 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:14.595623 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:14.599822 (+ 4199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:14.599832 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:14.599839 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:14.599891 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:14.603593 (+ 3702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:14.604567 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:14.604571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:14.604573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:14.604690 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:14.605212 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:14.605215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:14.605217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:14.605482 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:14.605590 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:14.606586 (+ 996us) server_negotiation.cc:299] Negotiation successful
0809 15:13:14.606853 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":235,"threads_started":1}
I20260809 15:13:14.608453 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:14.587671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38703 (local address 127.15.11.129:55013)
0809 15:13:14.588464 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:14.588501 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:14.590168 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:14.590896 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:14.590904 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:14.591387 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:14.591938 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:14.591960 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:14.593003 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:14.593007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:14.594811 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:14.594830 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:14.595767 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:14.595777 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:14.596021 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:14.596819 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:14.596851 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:14.599554 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:14.603734 (+ 4180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:14.603744 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:14.603763 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:14.604454 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:14.604820 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:14.604827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:14.604834 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:14.605087 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:14.605606 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:14.605616 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:14.606072 (+ 456us) client_negotiation.cc:769] Sending connection context
0809 15:13:14.606370 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 15:13:14.606940 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":280,"threads_started":1}
I20260809 15:13:14.610018 15669 heartbeater.cc:347] Connected to a master server at 127.15.11.190:38703
I20260809 15:13:14.610334 15669 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:14.611089 15669 heartbeater.cc:511] Master 127.15.11.190:38703 requested a full tablet report, sending...
I20260809 15:13:14.613262 15464 ts_manager.cc:194] Registered new tserver with Master: aad6ce9875e1449e9fb0628174bfe158 (127.15.11.129:34463)
I20260809 15:13:14.614701 15464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:55013
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:13:14.640873 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38703
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:14.963686 15676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:14.963975 15676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:14.964205 15676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:14.971095 15676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:14.971220 15676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:14.971401 15676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:13:14.980365 15676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:14.981451 15676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:14.982729 15676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:14.995704 15681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:14.995667 15684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:14.995734 15682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:14.996462 15676 server_base.cc:1034] running on GCE node
I20260809 15:13:14.997002 15676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:14.997833 15676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:14.999080 15676 hybrid_clock.cc:648] HybridClock initialized: now 1786288394999036 us; error 48 us; skew 500 ppm
I20260809 15:13:14.999435 15676 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:14 dist-test-slave-8w9v krb5kdc[15413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288394, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:15.002892 15676 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:13:15.004734 15676 webserver.cc:460] Webserver started at http://127.15.11.130:46711/ using document root <none> and password file <none>
I20260809 15:13:15.005476 15676 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:15.005637 15676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:15.005986 15676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:15.009157 15676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "af0c2e898ba1437bb71bb031f670edb1"
format_stamp: "Formatted at 2026-08-09 15:13:15 on dist-test-slave-8w9v"
server_key: "88c4cc1659357172458151ef80c5df79"
server_key_iv: "8b9ba6e590ab57a2c0151a11eaade913"
server_key_version: "encryptionkey@0"
I20260809 15:13:15.010022 15676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "af0c2e898ba1437bb71bb031f670edb1"
format_stamp: "Formatted at 2026-08-09 15:13:15 on dist-test-slave-8w9v"
server_key: "88c4cc1659357172458151ef80c5df79"
server_key_iv: "8b9ba6e590ab57a2c0151a11eaade913"
server_key_version: "encryptionkey@0"
I20260809 15:13:15.015548 15676 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:13:15.019119 15691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:15.020486 15676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:15.020671 15676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "af0c2e898ba1437bb71bb031f670edb1"
format_stamp: "Formatted at 2026-08-09 15:13:15 on dist-test-slave-8w9v"
server_key: "88c4cc1659357172458151ef80c5df79"
server_key_iv: "8b9ba6e590ab57a2c0151a11eaade913"
server_key_version: "encryptionkey@0"
I20260809 15:13:15.020884 15676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:15.033731 15676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:15.037602 15676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:15.037897 15676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:15.038863 15676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:15.040196 15676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:15.040316 15676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:15.040467 15676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:15.040539 15676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:15.074003 15676 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:34717
I20260809 15:13:15.074048 15809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:34717 every 8 connection(s)
I20260809 15:13:15.075317 15676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:13:15.076761 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 15676
I20260809 15:13:15.076982 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:13:15.077543 15406 external_mini_cluster.cc:1428] Setting key a2eee63c731f5b586fab7bc5aaeff553
Aug 09 15:13:15 dist-test-slave-8w9v krb5kdc[15413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288394, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:15.096876 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:15.078455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:57885 (local address 127.15.11.190:38703)
0809 15:13:15.078660 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:15.078666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:15.079615 (+ 949us) server_negotiation.cc:407] Connection header received
0809 15:13:15.082389 (+ 2774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:15.082393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:15.082466 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:15.082546 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:15.084545 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.085214 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.087288 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.087775 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.087892 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:15.091298 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:15.091305 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:15.091308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:15.091339 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:15.093997 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.094659 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.094663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.094682 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.094793 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.095235 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.095238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.095240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.095495 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:15.095654 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:15.096326 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 15:13:15.096558 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
I20260809 15:13:15.097774 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:15.078659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38703 (local address 127.15.11.130:57885)
0809 15:13:15.079465 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:15.079508 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:15.082090 (+ 2582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:15.082809 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:15.082823 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:15.083562 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:15.084348 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:15.084375 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.085358 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.085366 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:15.087079 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:15.087091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.087985 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.087990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:15.088241 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:15.088916 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:15.088937 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:15.091068 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:15.094121 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:15.094129 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:15.094142 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:15.094533 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:15.094910 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:15.094915 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:15.094918 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:15.095093 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:15.095615 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:15.095624 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:15.095929 (+ 305us) client_negotiation.cc:769] Sending connection context
0809 15:13:15.096235 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 15:13:15.096716 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":335,"threads_started":1}
I20260809 15:13:15.099145 15810 heartbeater.cc:347] Connected to a master server at 127.15.11.190:38703
I20260809 15:13:15.099454 15810 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
I20260809 15:13:15.100373 15810 heartbeater.cc:511] Master 127.15.11.190:38703 requested a full tablet report, sending...
I20260809 15:13:15.101836 15464 ts_manager.cc:194] Registered new tserver with Master: af0c2e898ba1437bb71bb031f670edb1 (127.15.11.130:34717)
I20260809 15:13:15.102641 15464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:57885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:13:15.122721 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38703
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:15.449880 15815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:15.450142 15815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:15.450328 15815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:15.457692 15815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:15.457824 15815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:15.458014 15815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:13:15.467729 15815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38703
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:15.469007 15815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:15.470362 15815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:15.483464 15821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:15.483536 15822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:15.483562 15824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:15.484454 15815 server_base.cc:1034] running on GCE node
I20260809 15:13:15.485059 15815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:15.485855 15815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:15.487053 15815 hybrid_clock.cc:648] HybridClock initialized: now 1786288395487014 us; error 29 us; skew 500 ppm
I20260809 15:13:15.487380 15815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:15 dist-test-slave-8w9v krb5kdc[15413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288395, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:15.490475 15815 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:13:15.491861 15815 webserver.cc:460] Webserver started at http://127.15.11.131:37049/ using document root <none> and password file <none>
I20260809 15:13:15.492655 15815 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:15.492806 15815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:15.493135 15815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:15.495983 15815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "92c8c9e5d99546fc83908f5e14355c79"
format_stamp: "Formatted at 2026-08-09 15:13:15 on dist-test-slave-8w9v"
server_key: "56bbce52b1704c04cef65d7eeb71b41b"
server_key_iv: "6ae93cc68086fc0abdbd9e93124b1a1a"
server_key_version: "encryptionkey@0"
I20260809 15:13:15.496811 15815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "92c8c9e5d99546fc83908f5e14355c79"
format_stamp: "Formatted at 2026-08-09 15:13:15 on dist-test-slave-8w9v"
server_key: "56bbce52b1704c04cef65d7eeb71b41b"
server_key_iv: "6ae93cc68086fc0abdbd9e93124b1a1a"
server_key_version: "encryptionkey@0"
I20260809 15:13:15.501822 15815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:13:15.505322 15831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:15.506518 15815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:15.506690 15815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "92c8c9e5d99546fc83908f5e14355c79"
format_stamp: "Formatted at 2026-08-09 15:13:15 on dist-test-slave-8w9v"
server_key: "56bbce52b1704c04cef65d7eeb71b41b"
server_key_iv: "6ae93cc68086fc0abdbd9e93124b1a1a"
server_key_version: "encryptionkey@0"
I20260809 15:13:15.506893 15815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:15.524269 15815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:15.528685 15815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:15.529011 15815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:15.530016 15815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:15.531388 15815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:15.531493 15815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:15.531639 15815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:15.531716 15815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:15.565855 15815 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:39257
I20260809 15:13:15.565898 15948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:39257 every 8 connection(s)
I20260809 15:13:15.567139 15815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:13:15.569931 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 15815
I20260809 15:13:15.570127 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:13:15.570708 15406 external_mini_cluster.cc:1428] Setting key 7c91e4789b5a662ee4dc7754c15b9e31
Aug 09 15:13:15 dist-test-slave-8w9v krb5kdc[15413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288395, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:15.588014 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:15.569743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:37399 (local address 127.15.11.190:38703)
0809 15:13:15.569920 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:15.569925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:15.571421 (+ 1496us) server_negotiation.cc:407] Connection header received
0809 15:13:15.573659 (+ 2238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:15.573664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:15.573827 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:15.573951 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:15.575426 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.576246 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.577826 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.578251 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.578386 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:15.581970 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:15.581977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:15.581981 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:15.582035 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:15.584756 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.585663 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.585668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.585672 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.585810 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.586321 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.586327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.586331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.586731 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:15.586886 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:15.587619 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 15:13:15.587772 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:13:15.589148 15951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:15.570399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38703 (local address 127.15.11.131:37399)
0809 15:13:15.571223 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:15.571267 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:15.573401 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:15.574200 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:15.574208 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:15.574705 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:15.575259 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:15.575275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.576406 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.576410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:15.577659 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:15.577671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.578488 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.578496 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:15.578723 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:15.579417 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:15.579446 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:15.581755 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:15.584892 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:15.584901 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:15.584917 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:15.585502 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:15.585932 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:15.585937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:15.585941 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:15.586203 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:15.586860 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:15.586868 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:15.587242 (+ 374us) client_negotiation.cc:769] Sending connection context
0809 15:13:15.587551 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 15:13:15.588036 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":365,"threads_started":1}
I20260809 15:13:15.590417 15949 heartbeater.cc:347] Connected to a master server at 127.15.11.190:38703
I20260809 15:13:15.590647 15949 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:15.591265 15949 heartbeater.cc:511] Master 127.15.11.190:38703 requested a full tablet report, sending...
I20260809 15:13:15.592617 15464 ts_manager.cc:194] Registered new tserver with Master: 92c8c9e5d99546fc83908f5e14355c79 (127.15.11.131:39257)
I20260809 15:13:15.593341 15464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:37399
I20260809 15:13:15.598611 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:13:15.617784 15669 heartbeater.cc:503] Master 127.15.11.190:38703 was elected leader, sending a full tablet report...
Aug 09 15:13:15 dist-test-slave-8w9v krb5kdc[15413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288395, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:13:15 dist-test-slave-8w9v krb5kdc[15413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288395, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:15.645567 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:15.631163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.15.11.129:34463)
0809 15:13:15.631799 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:15.631805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:15.631830 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:13:15.631962 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:15.631969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:15.632225 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:15.632360 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:15.633703 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.634863 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.636042 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.636641 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.636734 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:15.640420 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:15.640431 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:15.640444 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:15.640473 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:15.643280 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.643817 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.643824 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.643826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.643961 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.644307 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.644310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.644312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.644607 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:15.644815 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:15.644998 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 15:13:15.645169 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":353,"threads_started":1}
W20260809 15:13:15.646876 15579 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55966
Aug 09 15:13:15 dist-test-slave-8w9v krb5kdc[15413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:13:15 dist-test-slave-8w9v krb5kdc[15413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:15.690727 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:15.677918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.15.11.129:34463)
0809 15:13:15.678144 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:15.678150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:15.678381 (+ 231us) server_negotiation.cc:407] Connection header received
0809 15:13:15.678525 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:15.678529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:15.678618 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:15.678696 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:15.679905 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.680616 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.681721 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.682101 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.682183 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:15.685790 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:15.685795 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:15.685798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:15.685829 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:15.688575 (+ 2746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.689124 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.689128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.689130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.689239 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.689556 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.689559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.689561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.689804 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:15.689982 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:15.690213 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 15:13:15.690383 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:13:15.692200 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 15532
I20260809 15:13:15.704598 15956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:15.693247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55986 (local address 127.15.11.129:34463)
0809 15:13:15.693389 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:15.693393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:15.693729 (+ 336us) server_negotiation.cc:407] Connection header received
0809 15:13:15.693835 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:15.693838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:15.693886 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:15.693964 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:15.695061 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.695683 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.696992 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.697425 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.697513 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:15.699579 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:15.699583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:15.699586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:15.699611 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:15.702152 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.702962 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.702966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.702968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.703130 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.703449 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.703452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.703453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.703734 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:15.703909 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:15.704219 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 15:13:15.704355 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:13:15.705228 15664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:15.788221 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 15532
I20260809 15:13:15.809391 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 15676
Aug 09 15:13:15 dist-test-slave-8w9v krb5kdc[15413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:13:15.825429 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:15.810371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54476 (local address 127.15.11.130:34717)
0809 15:13:15.811041 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:15.811048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:15.811072 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:13:15.811131 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:15.811138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:15.811339 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:15.811460 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:15.812762 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.813719 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.814946 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.815454 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.815545 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:15.819659 (+ 4114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:15.819670 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:15.819683 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:15.819721 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:15.822512 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.823152 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.823159 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.823161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.823291 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.823644 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.823647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.823649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.823964 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:15.824344 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:15.824759 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 15:13:15.824993 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":384,"threads_started":1}
I20260809 15:13:15.826773 15804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:15.907469 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 15676
I20260809 15:13:15.928334 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 15815
Aug 09 15:13:15 dist-test-slave-8w9v krb5kdc[15413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:13:15.945245 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:15.929484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.15.11.131:39257)
0809 15:13:15.930207 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:15.930215 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:15.930251 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:13:15.930328 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:15.930335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:15.930557 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:15.930761 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:15.932422 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.933475 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.934864 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:15.935432 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:15.935575 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:15.939477 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:15.939489 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:15.939503 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:15.939548 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:15.942485 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.943177 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.943183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.943185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.943304 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:15.943615 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:15.943618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:15.943620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:15.943967 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:15.944285 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:15.944613 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 15:13:15.944918 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":382,"threads_started":1}
I20260809 15:13:15.946521 15943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:16.024631 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 15815
I20260809 15:13:16.044421 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 15428
I20260809 15:13:16.045620 15497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:16.145295 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 15428
2026-08-09T15:13:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3945 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:16 dist-test-slave-8w9v krb5kdc[15972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:16 dist-test-slave-8w9v krb5kdc[15972](info): set up 2 sockets
Aug 09 15:13:16 dist-test-slave-8w9v krb5kdc[15972](info): commencing operation
krb5kdc: starting...
W20260809 15:13:17.227654 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:17 dist-test-slave-8w9v krb5kdc[15972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:17.416982 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39559
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:39559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:17.736721 15986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:17.736999 15986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:17.737164 15986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:17.744529 15986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:17.744669 15986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:17.744799 15986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:17.744907 15986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:17.744982 15986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:17.757751 15986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:39559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:17.758965 15986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:17.760253 15986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:17.772205 15991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:17.772100 15994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:17.772164 15992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:17.773245 15986 server_base.cc:1034] running on GCE node
I20260809 15:13:17.774041 15986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:17.775444 15986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:17.776674 15986 hybrid_clock.cc:648] HybridClock initialized: now 1786288397776625 us; error 46 us; skew 500 ppm
I20260809 15:13:17.777009 15986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:17 dist-test-slave-8w9v krb5kdc[15972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288397, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:17.785434 15986 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:17.786839 15986 webserver.cc:460] Webserver started at http://127.15.11.190:36181/ using document root <none> and password file <none>
I20260809 15:13:17.787528 15986 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:17.787686 15986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:17.787990 15986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:17.790923 15986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "425d559dfedd466e85bac54696f9b710"
format_stamp: "Formatted at 2026-08-09 15:13:17 on dist-test-slave-8w9v"
server_key: "d4d04a52dae58b5205ce9995fa56f94c"
server_key_iv: "c17753421f266ed6f7e59c938aa54e9b"
server_key_version: "encryptionkey@0"
I20260809 15:13:17.791725 15986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "425d559dfedd466e85bac54696f9b710"
format_stamp: "Formatted at 2026-08-09 15:13:17 on dist-test-slave-8w9v"
server_key: "d4d04a52dae58b5205ce9995fa56f94c"
server_key_iv: "c17753421f266ed6f7e59c938aa54e9b"
server_key_version: "encryptionkey@0"
I20260809 15:13:17.796754 15986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:13:17.799938 16001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:17.801214 15986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:13:17.801412 15986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "425d559dfedd466e85bac54696f9b710"
format_stamp: "Formatted at 2026-08-09 15:13:17 on dist-test-slave-8w9v"
server_key: "d4d04a52dae58b5205ce9995fa56f94c"
server_key_iv: "c17753421f266ed6f7e59c938aa54e9b"
server_key_version: "encryptionkey@0"
I20260809 15:13:17.801612 15986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:17.814705 15986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:17.818353 15986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:17.818621 15986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:17.836786 15986 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:39559
I20260809 15:13:17.836818 16054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:39559 every 8 connection(s)
I20260809 15:13:17.838174 15986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:17.842041 16055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:17.845566 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 15986
I20260809 15:13:17.845825 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:17.846696 15406 external_mini_cluster.cc:1428] Setting key fefa6078f0cfa1782fe4b3bfd07cd366
I20260809 15:13:17.855197 16055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710: Bootstrap starting.
I20260809 15:13:17.859154 16055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:17.860556 16055 log.cc:824] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:13:17 dist-test-slave-8w9v krb5kdc[15972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:17.863598 16055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710: No bootstrap required, opened a new log
I20260809 15:13:17.870574 16058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:17.848748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.15.11.190:39559)
0809 15:13:17.849495 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:17.849506 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:17.849545 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:13:17.851171 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:17.851193 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:17.851564 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:17.851873 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:17.853735 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:17.855132 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:17.856983 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:17.857716 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:17.857867 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:17.863064 (+ 5197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:17.863073 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:17.863087 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:17.863124 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:17.866436 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:17.867145 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:17.867151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:17.867158 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:17.867325 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:17.867726 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:17.867730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:17.867732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:17.868336 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:17.868672 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:17.869177 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 15:13:17.869580 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":358,"threads_started":1}
I20260809 15:13:17.871619 16055 raft_consensus.cc:348] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425d559dfedd466e85bac54696f9b710" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39559 } }
I20260809 15:13:17.872030 16055 raft_consensus.cc:374] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:17.872162 16055 raft_consensus.cc:729] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 425d559dfedd466e85bac54696f9b710, State: Initialized, Role: FOLLOWER
I20260809 15:13:17.872750 16055 consensus_queue.cc:260] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425d559dfedd466e85bac54696f9b710" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39559 } }
I20260809 15:13:17.872998 16055 raft_consensus.cc:388] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:17.873176 16055 raft_consensus.cc:482] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:17.873377 16055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:17.875488 16055 raft_consensus.cc:504] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425d559dfedd466e85bac54696f9b710" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39559 } }
I20260809 15:13:17.875979 16055 leader_election.cc:302] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 425d559dfedd466e85bac54696f9b710; no voters:
I20260809 15:13:17.876585 16055 leader_election.cc:288] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:17.876681 16062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:17.877413 16062 raft_consensus.cc:686] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [term 1 LEADER]: Becoming Leader. State: Replica: 425d559dfedd466e85bac54696f9b710, State: Running, Role: LEADER
I20260809 15:13:17.877915 16062 consensus_queue.cc:237] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425d559dfedd466e85bac54696f9b710" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39559 } }
I20260809 15:13:17.878589 16055 sys_catalog.cc:564] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:17.880762 16063 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "425d559dfedd466e85bac54696f9b710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425d559dfedd466e85bac54696f9b710" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39559 } } }
I20260809 15:13:17.880867 16064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 425d559dfedd466e85bac54696f9b710. Latest consensus state: current_term: 1 leader_uuid: "425d559dfedd466e85bac54696f9b710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "425d559dfedd466e85bac54696f9b710" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39559 } } }
I20260809 15:13:17.881196 16063 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:17.881273 16064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:17.889252 16067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:17.895659 16067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:17.897481 16080 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 15:13:17.906109 16067 catalog_manager.cc:1347] Generated new cluster ID: e37dcb7c78824a5b8b1e3b57feedfc0f
I20260809 15:13:17.906225 16067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:17.947191 16067 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:17.949122 16067 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:17.961340 16067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 425d559dfedd466e85bac54696f9b710: Generated new TSK 0
I20260809 15:13:17.962128 16067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:13:18.021705 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39559
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:18.348754 16084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:18.349015 16084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:18.349103 16084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:18.355747 16084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:18.355855 16084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:18.355963 16084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:18.365245 16084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:18.366437 16084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:18.367959 16084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:18.381249 16091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:18.381278 16090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:18.381645 16093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:18.382588 16084 server_base.cc:1034] running on GCE node
I20260809 15:13:18.383131 16084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:18.384020 16084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:18.385272 16084 hybrid_clock.cc:648] HybridClock initialized: now 1786288398385237 us; error 63 us; skew 500 ppm
I20260809 15:13:18.385591 16084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:18 dist-test-slave-8w9v krb5kdc[15972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288398, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:18.388703 16084 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:13:18.390322 16084 webserver.cc:460] Webserver started at http://127.15.11.129:44685/ using document root <none> and password file <none>
I20260809 15:13:18.391073 16084 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:18.391263 16084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:18.391656 16084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:18.395529 16084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "5711cccd891d458f87e79bc2de0a9a51"
format_stamp: "Formatted at 2026-08-09 15:13:18 on dist-test-slave-8w9v"
server_key: "860b616d79d1854c19386b8b2e2812e4"
server_key_iv: "98675006e98b609226242b60ef7626d7"
server_key_version: "encryptionkey@0"
I20260809 15:13:18.396531 16084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "5711cccd891d458f87e79bc2de0a9a51"
format_stamp: "Formatted at 2026-08-09 15:13:18 on dist-test-slave-8w9v"
server_key: "860b616d79d1854c19386b8b2e2812e4"
server_key_iv: "98675006e98b609226242b60ef7626d7"
server_key_version: "encryptionkey@0"
I20260809 15:13:18.402666 16084 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 15:13:18.406405 16101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:18.407721 16084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:13:18.407899 16084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "5711cccd891d458f87e79bc2de0a9a51"
format_stamp: "Formatted at 2026-08-09 15:13:18 on dist-test-slave-8w9v"
server_key: "860b616d79d1854c19386b8b2e2812e4"
server_key_iv: "98675006e98b609226242b60ef7626d7"
server_key_version: "encryptionkey@0"
I20260809 15:13:18.408102 16084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:18.422801 16084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:18.426623 16084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:18.426940 16084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:18.427901 16084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:18.429266 16084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:18.429371 16084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:18.429545 16084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:18.429620 16084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:18.463693 16084 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:36097
I20260809 15:13:18.463732 16220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:36097 every 8 connection(s)
I20260809 15:13:18.465085 16084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:13:18.469528 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 16084
I20260809 15:13:18.469746 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:13:18.470589 15406 external_mini_cluster.cc:1428] Setting key ac214b4753fbaf66331241a1040238ce
Aug 09 15:13:18 dist-test-slave-8w9v krb5kdc[15972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288398, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:18.486668 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:18.467532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:36927 (local address 127.15.11.190:39559)
0809 15:13:18.468006 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:18.468013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:18.468994 (+ 981us) server_negotiation.cc:407] Connection header received
0809 15:13:18.471606 (+ 2612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:18.471612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:18.471693 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:18.471808 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:18.473583 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:18.474328 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:18.475940 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:18.476422 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:18.476565 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:18.480753 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:18.480758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:18.480761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:18.480793 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:18.483312 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:18.484245 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:18.484249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:18.484251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:18.484359 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:18.484826 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:18.484829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:18.484831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:18.485093 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:18.485202 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:18.486132 (+ 930us) server_negotiation.cc:299] Negotiation successful
0809 15:13:18.486349 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":215,"threads_started":1}
I20260809 15:13:18.488343 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:18.468084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39559 (local address 127.15.11.129:36927)
0809 15:13:18.468836 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:18.468876 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:18.471325 (+ 2449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:18.472098 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:18.472109 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:18.472777 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:18.473403 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:18.473421 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:18.474517 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:18.474521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:18.475751 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:18.475763 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:18.476658 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:18.476667 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:18.476935 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:18.477632 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:18.477656 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:18.480550 (+ 2894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:18.483454 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:18.483465 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:18.483482 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:18.484114 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:18.484467 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:18.484473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:18.484479 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:18.484718 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:18.485228 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:18.485239 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:18.485709 (+ 470us) client_negotiation.cc:769] Sending connection context
0809 15:13:18.486043 (+ 334us) client_negotiation.cc:240] Negotiation successful
0809 15:13:18.486656 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":263,"threads_started":1}
I20260809 15:13:18.490068 16221 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39559
I20260809 15:13:18.490379 16221 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
I20260809 15:13:18.491282 16221 heartbeater.cc:511] Master 127.15.11.190:39559 requested a full tablet report, sending...
I20260809 15:13:18.493520 16020 ts_manager.cc:194] Registered new tserver with Master: 5711cccd891d458f87e79bc2de0a9a51 (127.15.11.129:36097)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
I20260809 15:13:18.495115 16020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:36927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:13:18.516387 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39559
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:18.835672 16227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:18.835953 16227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:18.836148 16227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:18.843035 16227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:18.843127 16227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:18.843317 16227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:13:18.852500 16227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:18.853593 16227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:18.854875 16227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:18.868053 16232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:18.867969 16233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:18.868000 16235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:18.868814 16227 server_base.cc:1034] running on GCE node
I20260809 15:13:18.869415 16227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:18.870335 16227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:18.871572 16227 hybrid_clock.cc:648] HybridClock initialized: now 1786288398871539 us; error 60 us; skew 500 ppm
I20260809 15:13:18.871984 16227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:18 dist-test-slave-8w9v krb5kdc[15972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288398, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:18.875298 16227 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:13:18.876838 16227 webserver.cc:460] Webserver started at http://127.15.11.130:39795/ using document root <none> and password file <none>
I20260809 15:13:18.877542 16227 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:18.877684 16227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:18.878001 16227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:18.881198 16227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "0093493c07e34058954039c1bc073756"
format_stamp: "Formatted at 2026-08-09 15:13:18 on dist-test-slave-8w9v"
server_key: "6b4c2f6e1e3ee767824e351ba82466a3"
server_key_iv: "7c106236f77306205db71acdb76315fe"
server_key_version: "encryptionkey@0"
I20260809 15:13:18.881987 16227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "0093493c07e34058954039c1bc073756"
format_stamp: "Formatted at 2026-08-09 15:13:18 on dist-test-slave-8w9v"
server_key: "6b4c2f6e1e3ee767824e351ba82466a3"
server_key_iv: "7c106236f77306205db71acdb76315fe"
server_key_version: "encryptionkey@0"
I20260809 15:13:18.887022 16227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:13:18.890681 16242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:18.891985 16227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:13:18.892225 16227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "0093493c07e34058954039c1bc073756"
format_stamp: "Formatted at 2026-08-09 15:13:18 on dist-test-slave-8w9v"
server_key: "6b4c2f6e1e3ee767824e351ba82466a3"
server_key_iv: "7c106236f77306205db71acdb76315fe"
server_key_version: "encryptionkey@0"
I20260809 15:13:18.892477 16227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:18.905838 16227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:18.909725 16227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:18.910012 16227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:18.910912 16227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:18.912288 16227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:18.912387 16227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:18.912539 16227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:18.912621 16227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:18.944716 16227 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:32953
I20260809 15:13:18.944767 16361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:32953 every 8 connection(s)
I20260809 15:13:18.945986 16227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:13:18.953377 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 16227
I20260809 15:13:18.953567 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:13:18.954036 15406 external_mini_cluster.cc:1428] Setting key 416605443414cd4da8641f31820e4c89
Aug 09 15:13:18 dist-test-slave-8w9v krb5kdc[15972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288398, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:18.966464 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:18.948545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:51581 (local address 127.15.11.190:39559)
0809 15:13:18.948736 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:18.948741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:18.949856 (+ 1115us) server_negotiation.cc:407] Connection header received
0809 15:13:18.951661 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:18.951667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:18.951759 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:18.951871 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:18.953363 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:18.954233 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:18.956379 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:18.956795 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:18.956936 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:18.960622 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:18.960631 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:18.960639 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:18.960682 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:18.963508 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:18.964270 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:18.964279 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:18.964281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:18.964404 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:18.964863 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:18.964868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:18.964870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:18.965139 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:18.965293 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:18.965940 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 15:13:18.966160 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:13:18.967474 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:18.948996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39559 (local address 127.15.11.130:51581)
0809 15:13:18.949686 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:18.949725 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:18.951401 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:18.952097 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:18.952104 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:18.952652 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:18.953192 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:18.953208 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:18.954372 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:18.954378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:18.956240 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:18.956253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:18.957027 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:18.957034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:18.957269 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:18.958000 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:18.958021 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:18.960397 (+ 2376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:18.963670 (+ 3273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:18.963679 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:18.963693 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:18.964118 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:18.964536 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:18.964540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:18.964544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:18.964734 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:18.965269 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:18.965276 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:18.965587 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 15:13:18.965822 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 15:13:18.966295 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":263,"threads_started":1}
I20260809 15:13:18.969133 16362 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39559
I20260809 15:13:18.969421 16362 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:18.970121 16362 heartbeater.cc:511] Master 127.15.11.190:39559 requested a full tablet report, sending...
I20260809 15:13:18.971494 16020 ts_manager.cc:194] Registered new tserver with Master: 0093493c07e34058954039c1bc073756 (127.15.11.130:32953)
I20260809 15:13:18.972334 16020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:51581
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:13:18.995348 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39559
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:19.318241 16367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:19.318504 16367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:19.318608 16367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:19.325366 16367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:19.325466 16367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:19.325579 16367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:13:19.334864 16367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39559
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:19.336077 16367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:19.337391 16367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:19.351827 16372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:19.352010 16373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:19.352686 16375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:19.353168 16367 server_base.cc:1034] running on GCE node
I20260809 15:13:19.353716 16367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:19.354588 16367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:19.355818 16367 hybrid_clock.cc:648] HybridClock initialized: now 1786288399355759 us; error 55 us; skew 500 ppm
I20260809 15:13:19.356251 16367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:19 dist-test-slave-8w9v krb5kdc[15972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288399, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:19.359344 16367 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:13:19.360909 16367 webserver.cc:460] Webserver started at http://127.15.11.131:42987/ using document root <none> and password file <none>
I20260809 15:13:19.361647 16367 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:19.361809 16367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:19.362145 16367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:19.365105 16367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "b84739cc95f04bf9911b590ae6cc144e"
format_stamp: "Formatted at 2026-08-09 15:13:19 on dist-test-slave-8w9v"
server_key: "ec957991871ec641e0252a32db3d4a2d"
server_key_iv: "3388507e9f015434cb62522fe311132a"
server_key_version: "encryptionkey@0"
I20260809 15:13:19.365890 16367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "b84739cc95f04bf9911b590ae6cc144e"
format_stamp: "Formatted at 2026-08-09 15:13:19 on dist-test-slave-8w9v"
server_key: "ec957991871ec641e0252a32db3d4a2d"
server_key_iv: "3388507e9f015434cb62522fe311132a"
server_key_version: "encryptionkey@0"
I20260809 15:13:19.370872 16367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 15:13:19.374387 16382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:19.375630 16367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:13:19.375798 16367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "b84739cc95f04bf9911b590ae6cc144e"
format_stamp: "Formatted at 2026-08-09 15:13:19 on dist-test-slave-8w9v"
server_key: "ec957991871ec641e0252a32db3d4a2d"
server_key_iv: "3388507e9f015434cb62522fe311132a"
server_key_version: "encryptionkey@0"
I20260809 15:13:19.376008 16367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:19.396005 16367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:19.399760 16367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:19.400090 16367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:19.401094 16367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:19.402415 16367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:19.402518 16367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:19.402664 16367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:19.402736 16367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:19.438248 16367 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:36133
I20260809 15:13:19.438308 16500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:36133 every 8 connection(s)
I20260809 15:13:19.439579 16367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:13:19.443388 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 16367
I20260809 15:13:19.443585 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:13:19.444180 15406 external_mini_cluster.cc:1428] Setting key c6bf53bbad34ec6bca0f0018f1176007
Aug 09 15:13:19 dist-test-slave-8w9v krb5kdc[15972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288399, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:19.459764 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.442224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:46443 (local address 127.15.11.190:39559)
0809 15:13:19.442403 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.442408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.443685 (+ 1277us) server_negotiation.cc:407] Connection header received
0809 15:13:19.446021 (+ 2336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.446026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.446119 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.446222 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:19.447947 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.448710 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.450078 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.450515 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.450605 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.453865 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:19.453872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:19.453875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:19.453909 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:19.456694 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.457310 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.457313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.457315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.457439 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.457899 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.457910 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.457913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.458190 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:19.458352 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.459184 (+ 832us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.459419 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:13:19.460728 16503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.442632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39559 (local address 127.15.11.131:46443)
0809 15:13:19.443498 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:19.443558 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:19.445710 (+ 2152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:19.446485 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:19.446492 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:19.447021 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:19.447794 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.447814 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.448837 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.448840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.449955 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.449967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.450695 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.450699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.450880 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.451521 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:19.451545 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:19.453687 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:19.456805 (+ 3118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:19.456811 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:19.456822 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:19.457214 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:19.457546 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:19.457551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:19.457554 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:19.457784 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:19.458329 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:19.458338 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:19.458772 (+ 434us) client_negotiation.cc:769] Sending connection context
0809 15:13:19.459074 (+ 302us) client_negotiation.cc:240] Negotiation successful
0809 15:13:19.459588 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":308,"threads_started":1}
I20260809 15:13:19.462035 16501 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39559
I20260809 15:13:19.462343 16501 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:19.463104 16501 heartbeater.cc:511] Master 127.15.11.190:39559 requested a full tablet report, sending...
I20260809 15:13:19.464676 16020 ts_manager.cc:194] Registered new tserver with Master: b84739cc95f04bf9911b590ae6cc144e (127.15.11.131:36133)
I20260809 15:13:19.465751 16020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:46443
I20260809 15:13:19.471545 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:13:19.490162 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.477500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52862 (local address 127.15.11.190:39559)
0809 15:13:19.477682 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.477689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.478124 (+ 435us) server_negotiation.cc:407] Connection header received
0809 15:13:19.478339 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.478346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.478414 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.478516 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:19.480163 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.480961 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.482218 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.482587 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.482714 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.484952 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:19.484957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:19.484960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:19.484989 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:19.487409 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.488063 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.488069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.488073 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.488272 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.488657 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.488665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.488668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.488997 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:19.489264 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.489582 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.489812 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:13:19.497855 16221 heartbeater.cc:503] Master 127.15.11.190:39559 was elected leader, sending a full tablet report...
I20260809 15:13:19.496552 16020 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:13:19.503911 16020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:19.535404 16514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.515845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54858 (local address 127.15.11.131:36133)
0809 15:13:19.522946 (+ 7101us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.522958 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.523920 (+ 962us) server_negotiation.cc:407] Connection header received
0809 15:13:19.524356 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.524364 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.524617 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.524781 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:19.526224 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.528213 (+ 1989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.531645 (+ 3432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.533663 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.533792 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.534095 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.534765 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.534948 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6859,"thread_start_us":6671,"threads_started":1}
I20260809 15:13:19.535825 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.516617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.15.11.129:36097)
0809 15:13:19.523333 (+ 6716us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.523344 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.523912 (+ 568us) server_negotiation.cc:407] Connection header received
0809 15:13:19.524354 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.524362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.524618 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.524805 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:19.526348 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.528449 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.531919 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.533970 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.534107 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.534384 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.535139 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.535369 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6499,"thread_start_us":6345,"threads_started":1}
I20260809 15:13:19.537218 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.515706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:36133 (local address 127.0.0.1:54858)
0809 15:13:19.523767 (+ 8061us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:19.523816 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:19.524086 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:19.525212 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:19.525219 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:19.525271 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:19.525960 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.525975 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.528448 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.528457 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.531426 (+ 2969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.531446 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.534273 (+ 2827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.534279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.534472 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.534544 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 15:13:19.536237 (+ 1693us) client_negotiation.cc:240] Negotiation successful
0809 15:13:19.536546 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7792,"mutex_wait_us":80,"spinlock_wait_cycles":28544,"thread_start_us":7618,"threads_started":1}
I20260809 15:13:19.537592 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.516304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:36097 (local address 127.0.0.1:46270)
0809 15:13:19.523785 (+ 7481us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:19.523818 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:19.524088 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:19.525207 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:19.525217 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:19.525263 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:19.526099 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.526115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.528673 (+ 2558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.528680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.531702 (+ 3022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.531722 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.534634 (+ 2912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.534643 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.534893 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.534942 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 15:13:19.536293 (+ 1351us) client_negotiation.cc:240] Negotiation successful
0809 15:13:19.537015 (+ 722us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7345,"mutex_wait_us":590,"spinlock_wait_cycles":6272,"thread_start_us":7185,"threads_started":1}
I20260809 15:13:19.541606 16433 tablet_service.cc:1467] Processing CreateTablet for tablet d393b0cab91a4d9784cf10ae50551cc7 (DEFAULT_TABLE table=test-table [id=a58945bd5243483dbf29b0fb57cf1c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:19.541610 16154 tablet_service.cc:1467] Processing CreateTablet for tablet d393b0cab91a4d9784cf10ae50551cc7 (DEFAULT_TABLE table=test-table [id=a58945bd5243483dbf29b0fb57cf1c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:19.543139 16433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d393b0cab91a4d9784cf10ae50551cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:19.543151 16154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d393b0cab91a4d9784cf10ae50551cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:19.544034 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.524774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:32953 (local address 127.0.0.1:60078)
0809 15:13:19.537225 (+ 12451us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:19.537245 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:19.537375 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:19.538158 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:19.538163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:19.538188 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:19.538601 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.538612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.540411 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.540416 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.541729 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.541739 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.543320 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.543328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.543490 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.543512 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:13:19.543612 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 15:13:19.543731 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12279,"thread_start_us":12119,"threads_started":1}
I20260809 15:13:19.544202 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.524390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60078 (local address 127.15.11.130:32953)
0809 15:13:19.536832 (+ 12442us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.536843 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.537526 (+ 683us) server_negotiation.cc:407] Connection header received
0809 15:13:19.537654 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.537663 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.537954 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.538155 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:19.538777 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.540248 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.541949 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.543025 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.543178 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.543367 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.543674 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.543834 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12276,"thread_start_us":12135,"threads_started":1}
I20260809 15:13:19.547528 16294 tablet_service.cc:1467] Processing CreateTablet for tablet d393b0cab91a4d9784cf10ae50551cc7 (DEFAULT_TABLE table=test-table [id=a58945bd5243483dbf29b0fb57cf1c41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:19.548723 16294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d393b0cab91a4d9784cf10ae50551cc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:19.558085 16519 tablet_bootstrap.cc:492] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51: Bootstrap starting.
I20260809 15:13:19.561225 16520 tablet_bootstrap.cc:492] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e: Bootstrap starting.
I20260809 15:13:19.562706 16521 tablet_bootstrap.cc:492] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756: Bootstrap starting.
I20260809 15:13:19.563228 16519 tablet_bootstrap.cc:654] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:19.564402 16519 log.cc:824] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:19.564803 16520 tablet_bootstrap.cc:654] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:19.566004 16520 log.cc:824] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:19.566162 16521 tablet_bootstrap.cc:654] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:19.567271 16521 log.cc:824] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:19.567513 16519 tablet_bootstrap.cc:492] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51: No bootstrap required, opened a new log
I20260809 15:13:19.567821 16519 ts_tablet_manager.cc:1397] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 15:13:19.569563 16520 tablet_bootstrap.cc:492] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e: No bootstrap required, opened a new log
I20260809 15:13:19.569818 16520 ts_tablet_manager.cc:1397] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 15:13:19.570443 16521 tablet_bootstrap.cc:492] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756: No bootstrap required, opened a new log
I20260809 15:13:19.570691 16521 ts_tablet_manager.cc:1397] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 15:13:19.578532 16520 raft_consensus.cc:348] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } }
I20260809 15:13:19.579033 16520 raft_consensus.cc:374] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:19.578773 16519 raft_consensus.cc:348] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } }
I20260809 15:13:19.579134 16520 raft_consensus.cc:729] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84739cc95f04bf9911b590ae6cc144e, State: Initialized, Role: FOLLOWER
I20260809 15:13:19.579171 16519 raft_consensus.cc:374] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:19.579257 16519 raft_consensus.cc:729] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5711cccd891d458f87e79bc2de0a9a51, State: Initialized, Role: FOLLOWER
I20260809 15:13:19.579758 16520 consensus_queue.cc:260] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } }
I20260809 15:13:19.579965 16519 consensus_queue.cc:260] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } }
I20260809 15:13:19.581049 16501 heartbeater.cc:503] Master 127.15.11.190:39559 was elected leader, sending a full tablet report...
I20260809 15:13:19.581257 16521 raft_consensus.cc:348] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } }
I20260809 15:13:19.581666 16520 ts_tablet_manager.cc:1428] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260809 15:13:19.581753 16521 raft_consensus.cc:374] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:19.581862 16521 raft_consensus.cc:729] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0093493c07e34058954039c1bc073756, State: Initialized, Role: FOLLOWER
I20260809 15:13:19.582604 16521 consensus_queue.cc:260] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } }
I20260809 15:13:19.583464 16519 ts_tablet_manager.cc:1428] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51: Time spent starting tablet: real 0.015s user 0.017s sys 0.000s
I20260809 15:13:19.584504 16362 heartbeater.cc:503] Master 127.15.11.190:39559 was elected leader, sending a full tablet report...
I20260809 15:13:19.585094 16521 ts_tablet_manager.cc:1428] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
W20260809 15:13:19.691272 16502 tablet.cc:2369] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:13:19.697966 16363 tablet.cc:2369] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:13:19.717600 16222 tablet.cc:2369] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:19.761734 16527 raft_consensus.cc:482] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:19.762009 16527 raft_consensus.cc:504] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } }
I20260809 15:13:19.763672 16527 leader_election.cc:288] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b84739cc95f04bf9911b590ae6cc144e (127.15.11.131:36133), 0093493c07e34058954039c1bc073756 (127.15.11.130:32953)
I20260809 15:13:19.771241 16514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.764178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:48839 (local address 127.15.11.131:36133)
0809 15:13:19.764411 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.764420 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.764945 (+ 525us) server_negotiation.cc:407] Connection header received
0809 15:13:19.765178 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.765183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.765283 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.765380 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:19.766244 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.767278 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.769320 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.770355 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.770470 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.770521 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.770914 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.771001 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:13:19.771433 16529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.764185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:36133 (local address 127.15.11.129:48839)
0809 15:13:19.764821 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:19.764849 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:19.765010 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:19.765692 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:19.765696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:19.765730 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:19.766106 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.766116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.767408 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.767413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.769172 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.769184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.770558 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.770564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.770739 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.770778 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 15:13:19.770882 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:13:19.771010 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":690,"threads_started":1}
I20260809 15:13:19.772241 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.765313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:32953 (local address 127.15.11.129:49645)
0809 15:13:19.765996 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:19.766011 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:19.766158 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:19.766492 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:19.766496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:19.766522 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:19.766881 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.766893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.768110 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.768114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.769844 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:19.769859 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.771458 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.771467 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:19.771659 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.771699 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 15:13:19.771802 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 15:13:19.771924 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":331,"threads_started":1}
I20260809 15:13:19.772364 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.765524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:49645 (local address 127.15.11.130:32953)
0809 15:13:19.765754 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.765761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.766087 (+ 326us) server_negotiation.cc:407] Connection header received
0809 15:13:19.766323 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.766327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.766385 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.766447 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:19.767028 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.767968 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.770007 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.771071 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.771248 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.771314 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.771850 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.771952 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:13:19.772657 16454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393b0cab91a4d9784cf10ae50551cc7" candidate_uuid: "5711cccd891d458f87e79bc2de0a9a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84739cc95f04bf9911b590ae6cc144e" is_pre_election: true
I20260809 15:13:19.773221 16454 raft_consensus.cc:2454] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5711cccd891d458f87e79bc2de0a9a51 in term 0.
I20260809 15:13:19.773192 16316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393b0cab91a4d9784cf10ae50551cc7" candidate_uuid: "5711cccd891d458f87e79bc2de0a9a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0093493c07e34058954039c1bc073756" is_pre_election: true
I20260809 15:13:19.773795 16316 raft_consensus.cc:2454] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5711cccd891d458f87e79bc2de0a9a51 in term 0.
I20260809 15:13:19.773916 16107 leader_election.cc:302] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5711cccd891d458f87e79bc2de0a9a51, b84739cc95f04bf9911b590ae6cc144e; no voters:
I20260809 15:13:19.774326 16527 raft_consensus.cc:2781] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:19.774502 16527 raft_consensus.cc:482] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:19.774670 16527 raft_consensus.cc:3037] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:19.776876 16527 raft_consensus.cc:504] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } }
I20260809 15:13:19.777529 16527 leader_election.cc:288] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [CANDIDATE]: Term 1 election: Requested vote from peers b84739cc95f04bf9911b590ae6cc144e (127.15.11.131:36133), 0093493c07e34058954039c1bc073756 (127.15.11.130:32953)
I20260809 15:13:19.777853 16454 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393b0cab91a4d9784cf10ae50551cc7" candidate_uuid: "5711cccd891d458f87e79bc2de0a9a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b84739cc95f04bf9911b590ae6cc144e"
I20260809 15:13:19.777944 16316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393b0cab91a4d9784cf10ae50551cc7" candidate_uuid: "5711cccd891d458f87e79bc2de0a9a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0093493c07e34058954039c1bc073756"
I20260809 15:13:19.778029 16454 raft_consensus.cc:3037] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:19.778153 16316 raft_consensus.cc:3037] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:19.780443 16316 raft_consensus.cc:2454] T d393b0cab91a4d9784cf10ae50551cc7 P 0093493c07e34058954039c1bc073756 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5711cccd891d458f87e79bc2de0a9a51 in term 1.
I20260809 15:13:19.780582 16454 raft_consensus.cc:2454] T d393b0cab91a4d9784cf10ae50551cc7 P b84739cc95f04bf9911b590ae6cc144e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5711cccd891d458f87e79bc2de0a9a51 in term 1.
I20260809 15:13:19.780805 16107 leader_election.cc:302] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0093493c07e34058954039c1bc073756, 5711cccd891d458f87e79bc2de0a9a51; no voters:
I20260809 15:13:19.781080 16527 raft_consensus.cc:2781] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:19.781556 16527 raft_consensus.cc:686] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [term 1 LEADER]: Becoming Leader. State: Replica: 5711cccd891d458f87e79bc2de0a9a51, State: Running, Role: LEADER
I20260809 15:13:19.782130 16527 consensus_queue.cc:237] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } }
I20260809 15:13:19.786329 16018 catalog_manager.cc:5495] T d393b0cab91a4d9784cf10ae50551cc7 P 5711cccd891d458f87e79bc2de0a9a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5711cccd891d458f87e79bc2de0a9a51 (127.15.11.129). New cstate: current_term: 1 leader_uuid: "5711cccd891d458f87e79bc2de0a9a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b84739cc95f04bf9911b590ae6cc144e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 36133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0093493c07e34058954039c1bc073756" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 32953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5711cccd891d458f87e79bc2de0a9a51" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36097 } health_report { overall_health: HEALTHY } } }
Aug 09 15:13:19 dist-test-slave-8w9v krb5kdc[15972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:19.839020 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.824203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46286 (local address 127.15.11.129:36097)
0809 15:13:19.824417 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.824438 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.824908 (+ 470us) server_negotiation.cc:407] Connection header received
0809 15:13:19.825108 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.825114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.825183 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.825300 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:19.826863 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.827578 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.828931 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.829336 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.829479 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.833645 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:19.833670 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:19.833684 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:19.833715 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:19.836473 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.837218 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.837224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.837226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.837391 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.837737 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.837752 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.837754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.838043 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:19.838250 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.838520 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.838726 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 15:13:19 dist-test-slave-8w9v krb5kdc[15972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288399, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:13:19 dist-test-slave-8w9v krb5kdc[15972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288399, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:19.889487 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.876171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.15.11.129:36097)
0809 15:13:19.876440 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.876447 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.876786 (+ 339us) server_negotiation.cc:407] Connection header received
0809 15:13:19.876991 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.876995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.877063 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.877140 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:19.878650 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.879545 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.880919 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.881328 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.881421 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.884959 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:19.884964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:19.884966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:19.884998 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:19.887287 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.887810 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.887814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.887816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.887959 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.888365 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.888368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.888370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.888596 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:19.888809 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.888983 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.889139 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
Aug 09 15:13:19 dist-test-slave-8w9v krb5kdc[15972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:13:19 dist-test-slave-8w9v krb5kdc[15972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:19.936180 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.922545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.15.11.129:36097)
0809 15:13:19.922789 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.922806 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.922970 (+ 164us) server_negotiation.cc:407] Connection header received
0809 15:13:19.923171 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.923175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.923250 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.923351 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:19.924843 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.925625 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.926799 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.927204 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.927338 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.930991 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:19.930995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:19.930998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:19.931030 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:19.933622 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.934264 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.934271 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.934274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.934412 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.934762 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.934767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.934770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.935029 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:19.935295 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.935568 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.935786 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 15:13:19.938016 16133 tablet_service.cc:2908] Rejecting scan request for tablet d393b0cab91a4d9784cf10ae50551cc7: Uninitialized: clean time has not yet been initialized
I20260809 15:13:19.940927 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16084
I20260809 15:13:19.953747 16516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:19.941948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46320 (local address 127.15.11.129:36097)
0809 15:13:19.942090 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:19.942094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:19.942496 (+ 402us) server_negotiation.cc:407] Connection header received
0809 15:13:19.942626 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:19.942629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:19.942692 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:19.942807 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:19.943880 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.944640 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.945797 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:19.946175 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:19.946306 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:19.948579 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:19.948584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:19.948587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:19.948616 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:19.951143 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.951796 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.951806 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.951808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.951920 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:19.952354 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:19.952369 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:19.952373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:19.952628 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:19.952881 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:19.953187 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 15:13:19.953438 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:13:19.954536 16216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:20.042773 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 16084
I20260809 15:13:20.064465 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16227
Aug 09 15:13:20 dist-test-slave-8w9v krb5kdc[15972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:13:20.080631 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:20.065205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60080 (local address 127.15.11.130:32953)
0809 15:13:20.065401 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:20.065409 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:20.065479 (+ 70us) server_negotiation.cc:407] Connection header received
0809 15:13:20.065691 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:20.065695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:20.065755 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:20.065866 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:20.067076 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:20.067985 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:20.069442 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:20.069838 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:20.069977 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:20.074215 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:20.074228 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:20.074243 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:20.074276 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:20.077473 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:20.078102 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:20.078110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:20.078113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:20.078264 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:20.078662 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:20.078667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:20.078669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:20.079317 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:20.079568 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:20.079876 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 15:13:20.080094 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:13:20.081307 16357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:20.168258 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 16227
I20260809 15:13:20.189533 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16367
Aug 09 15:13:20 dist-test-slave-8w9v krb5kdc[15972](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288399, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:13:20.204191 16514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:20.190351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.15.11.131:36133)
0809 15:13:20.190544 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:20.190549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:20.190743 (+ 194us) server_negotiation.cc:407] Connection header received
0809 15:13:20.190907 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:20.190911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:20.190989 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:20.191090 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:20.192347 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:20.193242 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:20.194472 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:20.194831 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:20.194920 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:20.198942 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:20.198958 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:20.198974 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:20.199004 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:20.201837 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:20.202386 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:20.202393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:20.202395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:20.202521 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:20.202847 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:20.202850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:20.202852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:20.203184 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:20.203415 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:20.203597 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 15:13:20.203763 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:13:20.204823 16496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:20.289276 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 16367
I20260809 15:13:20.310724 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 15986
I20260809 15:13:20.311954 16050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:20.418576 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 15986
2026-08-09T15:13:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4273 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:20 dist-test-slave-8w9v krb5kdc[16550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:20 dist-test-slave-8w9v krb5kdc[16550](info): set up 2 sockets
Aug 09 15:13:20 dist-test-slave-8w9v krb5kdc[16550](info): commencing operation
krb5kdc: starting...
W20260809 15:13:21.525018 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.003s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:21 dist-test-slave-8w9v krb5kdc[16550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:21.718019 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35157
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:35157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:22.042752 16563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:22.043020 16563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:22.043185 16563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:22.050064 16563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:22.050174 16563 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 15:13:22.050218 16563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:22.050302 16563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:22.050372 16563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:22.050475 16563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:22.059777 16563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:35157
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:22.061031 16563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:22.062395 16563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:22.074330 16569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:22.074395 16572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:22.074419 16570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:22.075403 16563 server_base.cc:1034] running on GCE node
I20260809 15:13:22.076320 16563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:22.077657 16563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:22.078961 16563 hybrid_clock.cc:648] HybridClock initialized: now 1786288402078898 us; error 45 us; skew 500 ppm
I20260809 15:13:22.079326 16563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:22 dist-test-slave-8w9v krb5kdc[16550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288402, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:22.082449 16563 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:22.083922 16563 webserver.cc:460] Webserver started at http://127.15.11.190:36123/ using document root <none> and password file <none>
I20260809 15:13:22.084630 16563 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:22.084715 16563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:22.084981 16563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:22.087692 16563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "54e6f23f9be544d1b1b0497339ab1940"
format_stamp: "Formatted at 2026-08-09 15:13:22 on dist-test-slave-8w9v"
server_key: "e7f5e7916b17d39e1c4665162b302660"
server_key_iv: "b32015f1d62bc8da6c62fedd5fa4cb73"
server_key_version: "encryptionkey@0"
I20260809 15:13:22.088557 16563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "54e6f23f9be544d1b1b0497339ab1940"
format_stamp: "Formatted at 2026-08-09 15:13:22 on dist-test-slave-8w9v"
server_key: "e7f5e7916b17d39e1c4665162b302660"
server_key_iv: "b32015f1d62bc8da6c62fedd5fa4cb73"
server_key_version: "encryptionkey@0"
I20260809 15:13:22.093288 16563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:13:22.096616 16579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:22.097950 16563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:13:22.098142 16563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "54e6f23f9be544d1b1b0497339ab1940"
format_stamp: "Formatted at 2026-08-09 15:13:22 on dist-test-slave-8w9v"
server_key: "e7f5e7916b17d39e1c4665162b302660"
server_key_iv: "b32015f1d62bc8da6c62fedd5fa4cb73"
server_key_version: "encryptionkey@0"
I20260809 15:13:22.098374 16563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:22.129098 16563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:22.132798 16563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:22.133025 16563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:22.154573 16563 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:35157
I20260809 15:13:22.154594 16645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:35157 every 8 connection(s)
I20260809 15:13:22.155951 16563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:22.159641 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 16563
I20260809 15:13:22.159884 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:22.159999 16646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:22.160816 15406 external_mini_cluster.cc:1428] Setting key cddfcdbb413df9b4366c4f3c011a0c4a
I20260809 15:13:22.173480 16646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940: Bootstrap starting.
Aug 09 15:13:22 dist-test-slave-8w9v krb5kdc[16550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:22.177095 16646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:22.178128 16646 log.cc:824] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:22.181157 16646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940: No bootstrap required, opened a new log
I20260809 15:13:22.184345 16649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:22.162899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.15.11.190:35157)
0809 15:13:22.163739 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:22.163750 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:22.163790 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:13:22.166070 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:22.166092 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:22.166449 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:22.166744 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:22.168425 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.169671 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.171385 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.171993 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.172111 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:22.176848 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:22.176855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:22.176874 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:22.176908 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:22.180462 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:22.181108 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:22.181120 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:22.181126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:22.181335 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:22.181705 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:22.181710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:22.181713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:22.182203 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:22.182530 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:22.182920 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 15:13:22.183310 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":396,"threads_started":1}
I20260809 15:13:22.188724 16646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e6f23f9be544d1b1b0497339ab1940" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35157 } }
I20260809 15:13:22.189069 16646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:22.189180 16646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54e6f23f9be544d1b1b0497339ab1940, State: Initialized, Role: FOLLOWER
I20260809 15:13:22.189721 16646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e6f23f9be544d1b1b0497339ab1940" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35157 } }
I20260809 15:13:22.189978 16646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:22.190143 16646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:22.190327 16646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:22.192409 16646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e6f23f9be544d1b1b0497339ab1940" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35157 } }
I20260809 15:13:22.192869 16646 leader_election.cc:302] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54e6f23f9be544d1b1b0497339ab1940; no voters:
I20260809 15:13:22.193421 16646 leader_election.cc:288] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:22.193578 16651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:22.194809 16651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [term 1 LEADER]: Becoming Leader. State: Replica: 54e6f23f9be544d1b1b0497339ab1940, State: Running, Role: LEADER
I20260809 15:13:22.195297 16651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e6f23f9be544d1b1b0497339ab1940" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35157 } }
I20260809 15:13:22.195535 16646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:22.198148 16653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54e6f23f9be544d1b1b0497339ab1940. Latest consensus state: current_term: 1 leader_uuid: "54e6f23f9be544d1b1b0497339ab1940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e6f23f9be544d1b1b0497339ab1940" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35157 } } }
I20260809 15:13:22.198575 16653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:22.199112 16652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54e6f23f9be544d1b1b0497339ab1940" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e6f23f9be544d1b1b0497339ab1940" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35157 } } }
I20260809 15:13:22.199601 16652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:22.201269 16660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:22.206378 16660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:22.215268 16660 catalog_manager.cc:1347] Generated new cluster ID: 0be8ab6b200240e8a8ec8bbd97a1105b
I20260809 15:13:22.215426 16660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:22.234838 16660 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:22.236359 16660 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:22.244999 16660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940: Generated new TSK 0
I20260809 15:13:22.245975 16660 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:13:22.294677 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35157
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:22.623598 16674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:22.623847 16674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:22.623965 16674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:22.631009 16674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:22.631127 16674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:22.631311 16674 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:13:22.631407 16674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:22.641106 16674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.11.190:35157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:22.642427 16674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:22.643785 16674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:22.656675 16683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:22.656682 16679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:22.656701 16680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:22.662874 16674 server_base.cc:1034] running on GCE node
I20260809 15:13:22.663484 16674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:22.664428 16674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:22.665655 16674 hybrid_clock.cc:648] HybridClock initialized: now 1786288402665587 us; error 39 us; skew 500 ppm
I20260809 15:13:22.665982 16674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:22 dist-test-slave-8w9v krb5kdc[16550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288402, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:22.669138 16674 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:13:22.670670 16674 webserver.cc:460] Webserver started at http://127.15.11.129:35215/ using document root <none> and password file <none>
I20260809 15:13:22.671397 16674 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:22.671543 16674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:22.671867 16674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:22.674870 16674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "96094e0c3c31475197cc4f23a43dd092"
format_stamp: "Formatted at 2026-08-09 15:13:22 on dist-test-slave-8w9v"
server_key: "5e798e43e411373e40cd0fa61b7fddcc"
server_key_iv: "d2501b4ecf0c3b0c436fbb607ba94d26"
server_key_version: "encryptionkey@0"
I20260809 15:13:22.675669 16674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "96094e0c3c31475197cc4f23a43dd092"
format_stamp: "Formatted at 2026-08-09 15:13:22 on dist-test-slave-8w9v"
server_key: "5e798e43e411373e40cd0fa61b7fddcc"
server_key_iv: "d2501b4ecf0c3b0c436fbb607ba94d26"
server_key_version: "encryptionkey@0"
I20260809 15:13:22.680693 16674 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260809 15:13:22.684067 16690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:22.685551 16674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:22.685722 16674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "96094e0c3c31475197cc4f23a43dd092"
format_stamp: "Formatted at 2026-08-09 15:13:22 on dist-test-slave-8w9v"
server_key: "5e798e43e411373e40cd0fa61b7fddcc"
server_key_iv: "d2501b4ecf0c3b0c436fbb607ba94d26"
server_key_version: "encryptionkey@0"
I20260809 15:13:22.685982 16674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:22.705291 16674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:22.709038 16674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:22.709287 16674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:22.711793 16674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:22.711905 16674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:22.712064 16674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:22.712153 16674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:13:22 dist-test-slave-8w9v krb5kdc[16550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288402, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:22.735812 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:22.712502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:51047 (local address 127.15.11.190:35157)
0809 15:13:22.712969 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:22.712974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:22.714235 (+ 1261us) server_negotiation.cc:407] Connection header received
0809 15:13:22.716397 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:22.716403 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:22.716494 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:22.716614 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:22.718118 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.719045 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.722673 (+ 3628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.723204 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.723340 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:22.728333 (+ 4993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:22.728340 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:22.728348 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:22.728408 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:22.732164 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:22.733066 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:22.733074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:22.733080 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:22.733268 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:22.733719 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:22.733726 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:22.733732 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:22.734184 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:22.734384 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:22.734980 (+ 596us) server_negotiation.cc:299] Negotiation successful
0809 15:13:22.735275 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":258,"threads_started":1}
I20260809 15:13:22.736356 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:22.713268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35157 (local address 127.15.11.129:51047)
0809 15:13:22.714089 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:22.714128 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:22.716049 (+ 1921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:22.716879 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:22.716890 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:22.717382 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:22.717943 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:22.717960 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.719303 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.719311 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:22.722488 (+ 3177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:22.722506 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.723579 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.723590 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:22.724017 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:22.724808 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:22.724853 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:22.728078 (+ 3225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:22.732342 (+ 4264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:22.732359 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:22.732383 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:22.732907 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:22.733395 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:22.733398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:22.733400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:22.733593 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:22.734313 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:22.734318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:22.734620 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 15:13:22.734827 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 15:13:22.735162 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":371,"threads_started":1}
I20260809 15:13:22.757891 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:22.742134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:59425 (local address 127.15.11.190:35157)
0809 15:13:22.742333 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:22.742340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:22.743204 (+ 864us) server_negotiation.cc:407] Connection header received
0809 15:13:22.743415 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:22.743422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:22.743523 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:22.743629 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:22.745322 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.746313 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.748442 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.748814 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.748965 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:22.751640 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:22.751646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:22.751650 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:22.751689 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:22.754954 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:22.755932 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:22.755936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:22.755938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:22.756087 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:22.756580 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:22.756583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:22.756585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:22.756827 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:22.756940 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:22.757503 (+ 563us) server_negotiation.cc:299] Negotiation successful
0809 15:13:22.757660 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:13:22.758533 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:22.742509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35157 (local address 127.15.11.129:59425)
0809 15:13:22.743102 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:22.743120 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:22.743274 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:22.743779 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:22.743786 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:22.744572 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:22.745152 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:22.745173 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.746481 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.746485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:22.748246 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:22.748268 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:22.749274 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:22.749282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:22.749483 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:22.750188 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:22.750203 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:22.751386 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:22.755121 (+ 3735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:22.755129 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:22.755136 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:22.755813 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:22.756244 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:22.756252 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:22.756258 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:22.756466 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:22.756946 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:22.756954 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:22.757205 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 15:13:22.757409 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 15:13:22.757865 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":364,"threads_started":1}
I20260809 15:13:22.767391 16674 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:36987
I20260809 15:13:22.767423 16816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:36987 every 8 connection(s)
I20260809 15:13:22.768612 16674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:13:22.770481 16817 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35157
I20260809 15:13:22.770704 16817 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:22.771376 16817 heartbeater.cc:511] Master 127.15.11.190:35157 requested a full tablet report, sending...
I20260809 15:13:22.773412 16596 ts_manager.cc:194] Registered new tserver with Master: 96094e0c3c31475197cc4f23a43dd092 (127.15.11.129:36987)
I20260809 15:13:22.774474 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 16674
I20260809 15:13:22.774694 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:13:22.774914 16596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:51047
I20260809 15:13:22.775492 15406 external_mini_cluster.cc:1428] Setting key 7453a469ce3b1d146ae7258c3155f7e6
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:13:22.817411 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35157
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:23.137894 16821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:23.138217 16821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:23.138320 16821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:23.145411 16821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:23.145505 16821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:23.145591 16821 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:13:23.145668 16821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:13:23.155938 16821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.11.190:35157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:23.157467 16821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:23.159118 16821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:23.174115 16827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:23.174675 16829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:23.174192 16826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:23.175843 16821 server_base.cc:1034] running on GCE node
I20260809 15:13:23.176451 16821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:23.177297 16821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:23.178511 16821 hybrid_clock.cc:648] HybridClock initialized: now 1786288403178454 us; error 39 us; skew 500 ppm
I20260809 15:13:23.178746 16821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:23 dist-test-slave-8w9v krb5kdc[16550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288403, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:23.189291 16821 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:13:23.190696 16821 webserver.cc:460] Webserver started at http://127.15.11.130:36795/ using document root <none> and password file <none>
I20260809 15:13:23.191349 16821 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:23.191445 16821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:23.191723 16821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:23.194608 16821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "3783f8bad0a34e969f17ebcc3721eae1"
format_stamp: "Formatted at 2026-08-09 15:13:23 on dist-test-slave-8w9v"
server_key: "48170d0915c6325b4a8779bc6db96dc4"
server_key_iv: "774908bd74862dbc28ce7595b6cfcbca"
server_key_version: "encryptionkey@0"
I20260809 15:13:23.195375 16821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "3783f8bad0a34e969f17ebcc3721eae1"
format_stamp: "Formatted at 2026-08-09 15:13:23 on dist-test-slave-8w9v"
server_key: "48170d0915c6325b4a8779bc6db96dc4"
server_key_iv: "774908bd74862dbc28ce7595b6cfcbca"
server_key_version: "encryptionkey@0"
I20260809 15:13:23.200485 16821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 15:13:23.204080 16836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:23.205470 16821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:23.205644 16821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "3783f8bad0a34e969f17ebcc3721eae1"
format_stamp: "Formatted at 2026-08-09 15:13:23 on dist-test-slave-8w9v"
server_key: "48170d0915c6325b4a8779bc6db96dc4"
server_key_iv: "774908bd74862dbc28ce7595b6cfcbca"
server_key_version: "encryptionkey@0"
I20260809 15:13:23.205864 16821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:23.222409 16821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:23.226166 16821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:23.226481 16821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:23.229162 16821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:23.229297 16821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:23.229503 16821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:23.229583 16821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:13:23 dist-test-slave-8w9v krb5kdc[16550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288403, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:23.252856 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.229526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:36259 (local address 127.15.11.190:35157)
0809 15:13:23.229727 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:23.229735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:23.231241 (+ 1506us) server_negotiation.cc:407] Connection header received
0809 15:13:23.233423 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:23.233432 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:23.233570 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:23.233699 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:23.235937 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.237065 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.239903 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.240409 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.240551 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.245637 (+ 5086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:23.245642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:23.245652 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:23.245684 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:23.248433 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.249421 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.249429 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.249434 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.249637 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.250191 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.250199 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.250202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.250632 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:23.250798 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:23.251944 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0809 15:13:23.252317 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:13:23.253479 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.230148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35157 (local address 127.15.11.130:36259)
0809 15:13:23.231040 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:23.231087 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:23.233118 (+ 2031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:23.233979 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:23.233990 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:23.234734 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:23.235662 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.235696 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.237222 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.237232 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.239725 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.239751 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.240693 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.240705 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.241109 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.241989 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:23.242025 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:23.245424 (+ 3399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:23.248588 (+ 3164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:23.248604 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:23.248624 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:23.249261 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:23.249783 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:23.249789 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:23.249796 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:23.250059 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:23.250777 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:23.250790 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:23.251379 (+ 589us) client_negotiation.cc:769] Sending connection context
0809 15:13:23.251757 (+ 378us) client_negotiation.cc:240] Negotiation successful
0809 15:13:23.252266 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":381,"threads_started":1}
I20260809 15:13:23.278019 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.263034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35157 (local address 127.15.11.130:43281)
0809 15:13:23.263710 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:23.263734 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:23.263992 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:23.264486 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:23.264494 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:23.265018 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:23.265320 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.265329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.267100 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.267104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.268079 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.268090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.269057 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.269064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.269330 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.270028 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:23.270044 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:23.271024 (+ 980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:23.274711 (+ 3687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:23.274717 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:23.274721 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:23.275433 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:23.275842 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:23.275847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:23.275850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:23.275979 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:23.276655 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:23.276666 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:23.276956 (+ 290us) client_negotiation.cc:769] Sending connection context
0809 15:13:23.277164 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 15:13:23.277564 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":374,"threads_started":1}
I20260809 15:13:23.278124 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.263046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:43281 (local address 127.15.11.190:35157)
0809 15:13:23.263273 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:23.263282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:23.263856 (+ 574us) server_negotiation.cc:407] Connection header received
0809 15:13:23.264221 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:23.264229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:23.264322 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:23.264444 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:23.265475 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.266893 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.268276 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.268861 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.268992 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.271219 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:23.271224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:23.271227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:23.271256 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:23.274578 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.275572 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.275580 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.275583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.275731 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.276103 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.276108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.276112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.276507 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:23.276679 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:23.277544 (+ 865us) server_negotiation.cc:299] Negotiation successful
0809 15:13:23.277767 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:13:23.290086 16821 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:40903
I20260809 15:13:23.290115 16962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:40903 every 8 connection(s)
I20260809 15:13:23.291250 16821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:13:23.293324 16963 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35157
I20260809 15:13:23.293550 16963 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:23.294199 16963 heartbeater.cc:511] Master 127.15.11.190:35157 requested a full tablet report, sending...
I20260809 15:13:23.294682 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 16821
I20260809 15:13:23.294885 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:13:23.295473 15406 external_mini_cluster.cc:1428] Setting key 623d27233fec187160ad5396479347ee
I20260809 15:13:23.295635 16596 ts_manager.cc:194] Registered new tserver with Master: 3783f8bad0a34e969f17ebcc3721eae1 (127.15.11.130:40903)
I20260809 15:13:23.296649 16596 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:36259
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:13:23.339987 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35157
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:23.670065 16967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:23.670333 16967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:23.670439 16967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:23.677654 16967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:23.677752 16967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:23.677834 16967 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:13:23.677917 16967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:13:23.687351 16967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.11.190:35157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:23.688581 16967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:23.690029 16967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:23.703505 16973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:23.703531 16975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:23.704103 16972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:23.704697 16967 server_base.cc:1034] running on GCE node
I20260809 15:13:23.705269 16967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:23.706221 16967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:23.707468 16967 hybrid_clock.cc:648] HybridClock initialized: now 1786288403707404 us; error 43 us; skew 500 ppm
I20260809 15:13:23.707767 16967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:23 dist-test-slave-8w9v krb5kdc[16550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288403, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:23.711189 16967 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:13:23.712699 16967 webserver.cc:460] Webserver started at http://127.15.11.131:34065/ using document root <none> and password file <none>
I20260809 15:13:23.713399 16967 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:23.713534 16967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:23.713855 16967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:23.716748 16967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "c90942c07bde4509af9f1d7116a4a0b4"
format_stamp: "Formatted at 2026-08-09 15:13:23 on dist-test-slave-8w9v"
server_key: "1bd5d92e2ee2f0802dccd622dab31ae2"
server_key_iv: "4b0c6cda3ec8273b31994eb6d10bbbd7"
server_key_version: "encryptionkey@0"
I20260809 15:13:23.717631 16967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "c90942c07bde4509af9f1d7116a4a0b4"
format_stamp: "Formatted at 2026-08-09 15:13:23 on dist-test-slave-8w9v"
server_key: "1bd5d92e2ee2f0802dccd622dab31ae2"
server_key_iv: "4b0c6cda3ec8273b31994eb6d10bbbd7"
server_key_version: "encryptionkey@0"
I20260809 15:13:23.722534 16967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:13:23.725858 16983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:23.727047 16967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:13:23.727224 16967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "c90942c07bde4509af9f1d7116a4a0b4"
format_stamp: "Formatted at 2026-08-09 15:13:23 on dist-test-slave-8w9v"
server_key: "1bd5d92e2ee2f0802dccd622dab31ae2"
server_key_iv: "4b0c6cda3ec8273b31994eb6d10bbbd7"
server_key_version: "encryptionkey@0"
I20260809 15:13:23.727439 16967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:23.751888 16967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:23.756091 16967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:23.756464 16967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:23.759363 16967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:23.759486 16967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:23.759639 16967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:23.759707 16967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:13:23 dist-test-slave-8w9v krb5kdc[16550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288403, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:23.777995 16817 heartbeater.cc:503] Master 127.15.11.190:35157 was elected leader, sending a full tablet report...
I20260809 15:13:23.781759 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.759850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:34281 (local address 127.15.11.190:35157)
0809 15:13:23.760037 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:23.760041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:23.761455 (+ 1414us) server_negotiation.cc:407] Connection header received
0809 15:13:23.763441 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:23.763446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:23.763539 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:23.763647 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:23.765831 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.766969 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.769412 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.769840 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.769977 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.773700 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:23.773707 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:23.773714 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:23.773764 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:23.777108 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.778132 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.778137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.778140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.778338 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.778934 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.778943 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.778950 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.779442 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:23.779626 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:23.781280 (+ 1654us) server_negotiation.cc:299] Negotiation successful
0809 15:13:23.781502 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:13:23.783432 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.760499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35157 (local address 127.15.11.131:34281)
0809 15:13:23.761272 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:23.761316 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:23.763177 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:23.763895 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:23.763903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:23.764670 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:23.765583 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.765617 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.767127 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.767135 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.769263 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.769276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.770083 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.770091 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.770296 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.771006 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:23.771027 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:23.773460 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:23.777255 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:23.777266 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:23.777286 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:23.777851 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:23.778495 (+ 644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:23.778502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:23.778509 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:23.778740 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:23.779722 (+ 982us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:23.779734 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:23.780358 (+ 624us) client_negotiation.cc:769] Sending connection context
0809 15:13:23.780738 (+ 380us) client_negotiation.cc:240] Negotiation successful
0809 15:13:23.781461 (+ 723us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":349,"threads_started":1}
I20260809 15:13:23.805802 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.789234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:36661 (local address 127.15.11.190:35157)
0809 15:13:23.789439 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:23.789446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:23.790244 (+ 798us) server_negotiation.cc:407] Connection header received
0809 15:13:23.790430 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:23.790434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:23.790491 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:23.790604 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:23.792280 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.793166 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.795613 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.796261 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.796405 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.799165 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:23.799173 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:23.799178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:23.799223 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:23.802947 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.803811 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.803816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.803821 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.803968 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.804369 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.804376 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.804379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.804651 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:23.804798 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:23.805237 (+ 439us) server_negotiation.cc:299] Negotiation successful
0809 15:13:23.805454 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:13:23.805922 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.789303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35157 (local address 127.15.11.131:36661)
0809 15:13:23.790127 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:23.790151 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:23.790312 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:23.790704 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:23.790712 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:23.791433 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:23.792003 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.792024 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.793316 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.793325 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.795415 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.795435 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.796479 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.796488 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.796770 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.797590 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:23.797609 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:23.798978 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:23.803095 (+ 4117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:23.803100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:23.803103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:23.803629 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:23.804101 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:23.804105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:23.804109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:23.804263 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:23.804768 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:23.804773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:23.804970 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 15:13:23.805120 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 15:13:23.805460 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":424,"threads_started":1}
I20260809 15:13:23.824620 16967 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:46593
I20260809 15:13:23.824648 17109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:46593 every 8 connection(s)
I20260809 15:13:23.825767 16967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:13:23.827816 17110 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35157
I20260809 15:13:23.828053 17110 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:23.828728 17110 heartbeater.cc:511] Master 127.15.11.190:35157 requested a full tablet report, sending...
I20260809 15:13:23.829134 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 16967
I20260809 15:13:23.829339 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:13:23.829861 15406 external_mini_cluster.cc:1428] Setting key 31fff30404c8daaa07e6fc08f09930c8
I20260809 15:13:23.830261 16595 ts_manager.cc:194] Registered new tserver with Master: c90942c07bde4509af9f1d7116a4a0b4 (127.15.11.131:46593)
I20260809 15:13:23.831359 16595 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:34281
I20260809 15:13:23.833032 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:13:23 dist-test-slave-8w9v krb5kdc[16550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288403, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:13:23 dist-test-slave-8w9v krb5kdc[16550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288403, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:23.880694 16699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.867598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.15.11.190:35157)
0809 15:13:23.867833 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:23.867841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:23.867982 (+ 141us) server_negotiation.cc:407] Connection header received
0809 15:13:23.868183 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:23.868187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:23.868266 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:23.868362 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:23.869680 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.870395 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.871574 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.871978 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.872063 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.875757 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:23.875762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:23.875765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:23.875796 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:23.878308 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.878925 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.878929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.878931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.879064 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:23.879404 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:23.879406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:23.879408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:23.879635 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:23.879796 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:23.880080 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 15:13:23.880290 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 15:13:23.884955 16595 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:13:23.892598 16595 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:23.920898 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.904690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:40903 (local address 127.0.0.1:46574)
0809 15:13:23.905464 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:23.905509 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:23.905724 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:23.907712 (+ 1988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:23.907723 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:23.909014 (+ 1291us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:23.909603 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.909620 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.913572 (+ 3952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.913580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.918323 (+ 4743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.918360 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.919694 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.919700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.919928 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.919999 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 15:13:23.920227 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 15:13:23.920446 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":285,"threads_started":1}
I20260809 15:13:23.921954 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.904916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.15.11.130:40903)
0809 15:13:23.906609 (+ 1693us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:23.906621 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:23.906658 (+ 37us) server_negotiation.cc:407] Connection header received
0809 15:13:23.906793 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:23.906804 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:23.907117 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:23.907364 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:23.909938 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.913359 (+ 3421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.918573 (+ 5214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.919488 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.919660 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.919912 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:23.921474 (+ 1562us) server_negotiation.cc:299] Negotiation successful
0809 15:13:23.921654 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":898,"thread_start_us":753,"threads_started":1}
I20260809 15:13:23.925887 16890 tablet_service.cc:1467] Processing CreateTablet for tablet 20783f5981b447118e721e9b7ac93008 (DEFAULT_TABLE table=test-table [id=01eb300cae8a4ad98e723a713854c357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:23.927158 16890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20783f5981b447118e721e9b7ac93008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:23.929855 17126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.906048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:46593 (local address 127.0.0.1:47086)
0809 15:13:23.918681 (+ 12633us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:23.918705 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:23.918844 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:23.922677 (+ 3833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:23.922681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:23.922706 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:23.923250 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.923260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.924745 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.924752 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.927140 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.927155 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.928996 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.929003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.929180 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.929218 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:13:23.929336 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 15:13:23.929462 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12467,"mutex_wait_us":24,"thread_start_us":12277,"threads_started":1}
I20260809 15:13:23.929917 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.906291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.15.11.131:46593)
0809 15:13:23.921712 (+ 15421us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:23.921720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:23.921755 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:13:23.921850 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:23.921858 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:23.922164 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:23.922371 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:23.923459 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.924606 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.927336 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.928782 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.928933 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.929175 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:23.929394 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 15:13:23.929552 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15253,"thread_start_us":15070,"threads_started":1}
I20260809 15:13:23.931689 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.906645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:36987 (local address 127.0.0.1:53518)
0809 15:13:23.920852 (+ 14207us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:23.920886 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:23.921035 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:23.922369 (+ 1334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:23.922374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:23.922413 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:23.922957 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.922980 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.924882 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.924888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.928277 (+ 3389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:23.928292 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.930515 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.930523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:23.930697 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.930732 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 15:13:23.931168 (+ 436us) client_negotiation.cc:240] Negotiation successful
0809 15:13:23.931310 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14089,"mutex_wait_us":72,"thread_start_us":13931,"threads_started":1}
I20260809 15:13:23.932536 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:23.907193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53518 (local address 127.15.11.129:36987)
0809 15:13:23.920396 (+ 13203us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:23.920405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:23.921002 (+ 597us) server_negotiation.cc:407] Connection header received
0809 15:13:23.921907 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:23.921915 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:23.922158 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:23.922387 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:23.923182 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.924731 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.928449 (+ 3718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:23.929829 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:23.930857 (+ 1028us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:23.931048 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:23.932029 (+ 981us) server_negotiation.cc:299] Negotiation successful
0809 15:13:23.932227 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13045,"thread_start_us":12914,"threads_started":1}
I20260809 15:13:23.934182 17040 tablet_service.cc:1467] Processing CreateTablet for tablet 20783f5981b447118e721e9b7ac93008 (DEFAULT_TABLE table=test-table [id=01eb300cae8a4ad98e723a713854c357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:23.935806 16743 tablet_service.cc:1467] Processing CreateTablet for tablet 20783f5981b447118e721e9b7ac93008 (DEFAULT_TABLE table=test-table [id=01eb300cae8a4ad98e723a713854c357]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:23.936635 17040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20783f5981b447118e721e9b7ac93008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:23.937237 16743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20783f5981b447118e721e9b7ac93008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:23.939851 17131 tablet_bootstrap.cc:492] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: Bootstrap starting.
I20260809 15:13:23.942656 17131 tablet_bootstrap.cc:654] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:23.943787 17131 log.cc:824] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:23.946249 17131 tablet_bootstrap.cc:492] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: No bootstrap required, opened a new log
I20260809 15:13:23.946542 17131 ts_tablet_manager.cc:1397] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:13:23.950816 17133 tablet_bootstrap.cc:492] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: Bootstrap starting.
I20260809 15:13:23.954483 17133 tablet_bootstrap.cc:654] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:23.954658 17131 raft_consensus.cc:348] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } }
I20260809 15:13:23.955168 17131 raft_consensus.cc:374] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:23.955320 17131 raft_consensus.cc:729] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3783f8bad0a34e969f17ebcc3721eae1, State: Initialized, Role: FOLLOWER
I20260809 15:13:23.956009 17133 log.cc:824] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:23.955945 17131 consensus_queue.cc:260] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } }
I20260809 15:13:23.957562 16963 heartbeater.cc:503] Master 127.15.11.190:35157 was elected leader, sending a full tablet report...
I20260809 15:13:23.957837 17134 tablet_bootstrap.cc:492] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: Bootstrap starting.
I20260809 15:13:23.958263 17131 ts_tablet_manager.cc:1428] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260809 15:13:23.960305 17133 tablet_bootstrap.cc:492] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: No bootstrap required, opened a new log
I20260809 15:13:23.960585 17133 ts_tablet_manager.cc:1397] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20260809 15:13:23.961422 17134 tablet_bootstrap.cc:654] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:23.962518 17134 log.cc:824] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:23.966017 17134 tablet_bootstrap.cc:492] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: No bootstrap required, opened a new log
I20260809 15:13:23.966275 17134 ts_tablet_manager.cc:1397] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 15:13:23.968698 17133 raft_consensus.cc:348] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } }
I20260809 15:13:23.969056 17133 raft_consensus.cc:374] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:23.969158 17133 raft_consensus.cc:729] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96094e0c3c31475197cc4f23a43dd092, State: Initialized, Role: FOLLOWER
I20260809 15:13:23.969691 17133 consensus_queue.cc:260] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } }
I20260809 15:13:23.971165 17133 ts_tablet_manager.cc:1428] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
I20260809 15:13:23.974836 17134 raft_consensus.cc:348] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } }
I20260809 15:13:23.975209 17134 raft_consensus.cc:374] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:23.975289 17134 raft_consensus.cc:729] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90942c07bde4509af9f1d7116a4a0b4, State: Initialized, Role: FOLLOWER
I20260809 15:13:23.975790 17134 consensus_queue.cc:260] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } }
I20260809 15:13:23.976809 17110 heartbeater.cc:503] Master 127.15.11.190:35157 was elected leader, sending a full tablet report...
I20260809 15:13:23.977437 17134 ts_tablet_manager.cc:1428] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: Time spent starting tablet: real 0.011s user 0.003s sys 0.009s
W20260809 15:13:24.021277 16818 tablet.cc:2369] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:24.031534 17139 raft_consensus.cc:482] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:24.031898 17139 raft_consensus.cc:504] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } }
I20260809 15:13:24.033690 17139 leader_election.cc:288] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3783f8bad0a34e969f17ebcc3721eae1 (127.15.11.130:40903), 96094e0c3c31475197cc4f23a43dd092 (127.15.11.129:36987)
I20260809 15:13:24.040345 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.034279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:55193 (local address 127.15.11.129:36987)
0809 15:13:24.034510 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.034519 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.034544 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:13:24.034641 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.034652 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.034733 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.034841 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:24.035300 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.036268 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.038337 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.039401 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.039557 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.039632 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.039942 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.040095 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:13:24.040366 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.033953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:36987 (local address 127.15.11.131:55193)
0809 15:13:24.034303 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:24.034327 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:24.034451 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:24.034889 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:24.034894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:24.034920 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:24.035164 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.035178 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.036401 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.036405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.038174 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.038188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.039537 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.039541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.039778 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.039805 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:13:24.039881 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 15:13:24.039977 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260809 15:13:24.040992 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.034029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:42851 (local address 127.15.11.130:40903)
0809 15:13:24.034279 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.034285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.035099 (+ 814us) server_negotiation.cc:407] Connection header received
0809 15:13:24.035267 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.035274 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.035353 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.035468 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:24.035964 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.036787 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.038800 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.039975 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.040147 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.040215 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.040639 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.040770 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260809 15:13:24.041512 16770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20783f5981b447118e721e9b7ac93008" candidate_uuid: "c90942c07bde4509af9f1d7116a4a0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96094e0c3c31475197cc4f23a43dd092" is_pre_election: true
I20260809 15:13:24.041759 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.034101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:40903 (local address 127.15.11.131:42851)
0809 15:13:24.034989 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:24.035006 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:24.035128 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:24.035492 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:24.035495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:24.035514 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:24.035805 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.035812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.036927 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.036931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.038664 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.038675 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.040237 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.040243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.040413 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.040449 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 15:13:24.041203 (+ 754us) client_negotiation.cc:240] Negotiation successful
0809 15:13:24.041331 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"mutex_wait_us":62,"thread_start_us":445,"threads_started":1}
I20260809 15:13:24.042150 16770 raft_consensus.cc:2454] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c90942c07bde4509af9f1d7116a4a0b4 in term 0.
I20260809 15:13:24.042825 16987 leader_election.cc:302] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96094e0c3c31475197cc4f23a43dd092, c90942c07bde4509af9f1d7116a4a0b4; no voters:
I20260809 15:13:24.042865 16916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20783f5981b447118e721e9b7ac93008" candidate_uuid: "c90942c07bde4509af9f1d7116a4a0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3783f8bad0a34e969f17ebcc3721eae1" is_pre_election: true
W20260809 15:13:24.043272 16964 tablet.cc:2369] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:24.043326 17139 raft_consensus.cc:2781] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:24.043562 17139 raft_consensus.cc:482] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:24.043485 16916 raft_consensus.cc:2454] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c90942c07bde4509af9f1d7116a4a0b4 in term 0.
I20260809 15:13:24.043676 17139 raft_consensus.cc:3037] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:24.046115 17139 raft_consensus.cc:504] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } }
I20260809 15:13:24.046850 17139 leader_election.cc:288] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [CANDIDATE]: Term 1 election: Requested vote from peers 3783f8bad0a34e969f17ebcc3721eae1 (127.15.11.130:40903), 96094e0c3c31475197cc4f23a43dd092 (127.15.11.129:36987)
I20260809 15:13:24.047269 16916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20783f5981b447118e721e9b7ac93008" candidate_uuid: "c90942c07bde4509af9f1d7116a4a0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3783f8bad0a34e969f17ebcc3721eae1"
I20260809 15:13:24.047510 16916 raft_consensus.cc:3037] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:24.047534 16770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20783f5981b447118e721e9b7ac93008" candidate_uuid: "c90942c07bde4509af9f1d7116a4a0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96094e0c3c31475197cc4f23a43dd092"
I20260809 15:13:24.047780 16770 raft_consensus.cc:3037] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:24.049793 16916 raft_consensus.cc:2454] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c90942c07bde4509af9f1d7116a4a0b4 in term 1.
I20260809 15:13:24.050163 16986 leader_election.cc:302] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3783f8bad0a34e969f17ebcc3721eae1, c90942c07bde4509af9f1d7116a4a0b4; no voters:
I20260809 15:13:24.050225 16770 raft_consensus.cc:2454] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c90942c07bde4509af9f1d7116a4a0b4 in term 1.
I20260809 15:13:24.050501 17139 raft_consensus.cc:2781] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:24.050979 17139 raft_consensus.cc:686] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 1 LEADER]: Becoming Leader. State: Replica: c90942c07bde4509af9f1d7116a4a0b4, State: Running, Role: LEADER
I20260809 15:13:24.051478 17139 consensus_queue.cc:237] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } }
I20260809 15:13:24.056350 16595 catalog_manager.cc:5495] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to c90942c07bde4509af9f1d7116a4a0b4 (127.15.11.131). New cstate: current_term: 1 leader_uuid: "c90942c07bde4509af9f1d7116a4a0b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } health_report { overall_health: UNKNOWN } } }
W20260809 15:13:24.077389 17111 tablet.cc:2369] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:24.114240 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.108779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.15.11.131:46593)
0809 15:13:24.108979 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.108983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.109112 (+ 129us) server_negotiation.cc:407] Connection header received
0809 15:13:24.109477 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.109480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.109554 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.109671 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:24.110046 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.110817 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.112198 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.112608 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.112698 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.113238 (+ 540us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:24.113721 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:24.113820 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.113953 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.114038 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:13:24.122864 16916 raft_consensus.cc:1261] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Refusing update from remote peer c90942c07bde4509af9f1d7116a4a0b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:24.122892 16770 raft_consensus.cc:1261] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [term 1 FOLLOWER]: Refusing update from remote peer c90942c07bde4509af9f1d7116a4a0b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:24.123869 17141 consensus_queue.cc:1035] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:24.124316 17139 consensus_queue.cc:1035] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:24.156265 16596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:24.156597 16596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:24.160962 16596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940: Sending DeleteTablet for 3 replicas of tablet 20783f5981b447118e721e9b7ac93008
I20260809 15:13:24.162230 16890 tablet_service.cc:1514] Processing DeleteTablet for tablet 20783f5981b447118e721e9b7ac93008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:24 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:46574
I20260809 15:13:24.162266 16743 tablet_service.cc:1514] Processing DeleteTablet for tablet 20783f5981b447118e721e9b7ac93008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:24 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:53518
I20260809 15:13:24.165959 17040 tablet_service.cc:1514] Processing DeleteTablet for tablet 20783f5981b447118e721e9b7ac93008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:24 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:47086
I20260809 15:13:24.165990 16596 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43924:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:13:24.167567 16596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:24.177623 16742 tablet_service.cc:1467] Processing CreateTablet for tablet 82da1572dda04affac5871d01a3edd3f (DEFAULT_TABLE table=test-table [id=045e47a9ebfa478d9f9687af7c2edbd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:24.178292 16742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82da1572dda04affac5871d01a3edd3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:24.178436 16889 tablet_service.cc:1467] Processing CreateTablet for tablet 82da1572dda04affac5871d01a3edd3f (DEFAULT_TABLE table=test-table [id=045e47a9ebfa478d9f9687af7c2edbd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:24.179010 16889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82da1572dda04affac5871d01a3edd3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:24.180209 17155 tablet_replica.cc:330] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: stopping tablet replica
I20260809 15:13:24.180836 17155 raft_consensus.cc:2229] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:24.180979 17156 tablet_replica.cc:330] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: stopping tablet replica
I20260809 15:13:24.181439 17155 raft_consensus.cc:2258] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:24.181448 17156 raft_consensus.cc:2229] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:13:24.182183 17156 raft_consensus.cc:2258] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:24.183939 17155 ts_tablet_manager.cc:1905] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:24.184844 17040 tablet_service.cc:1467] Processing CreateTablet for tablet 82da1572dda04affac5871d01a3edd3f (DEFAULT_TABLE table=test-table [id=045e47a9ebfa478d9f9687af7c2edbd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:24.185285 17040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82da1572dda04affac5871d01a3edd3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:24.188097 17133 tablet_bootstrap.cc:492] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: Bootstrap starting.
I20260809 15:13:24.189905 17133 tablet_bootstrap.cc:654] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:24.191004 17131 tablet_bootstrap.cc:492] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: Bootstrap starting.
I20260809 15:13:24.191308 17156 ts_tablet_manager.cc:1905] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:24.192703 17131 tablet_bootstrap.cc:654] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:24.194211 17134 tablet_bootstrap.cc:492] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: Bootstrap starting.
I20260809 15:13:24.196343 17134 tablet_bootstrap.cc:654] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:24.198839 17155 ts_tablet_manager.cc:1918] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:13:24.199093 17155 log.cc:1196] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/20783f5981b447118e721e9b7ac93008
I20260809 15:13:24.199821 17155 ts_tablet_manager.cc:1939] T 20783f5981b447118e721e9b7ac93008 P 96094e0c3c31475197cc4f23a43dd092: Deleting consensus metadata
I20260809 15:13:24.201885 17131 tablet_bootstrap.cc:492] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: No bootstrap required, opened a new log
I20260809 15:13:24.202081 17131 ts_tablet_manager.cc:1397] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260809 15:13:24.203014 17131 raft_consensus.cc:348] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } }
I20260809 15:13:24.203289 17131 raft_consensus.cc:374] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:24.203287 16583 catalog_manager.cc:4841] TS 96094e0c3c31475197cc4f23a43dd092 (127.15.11.129:36987): tablet 20783f5981b447118e721e9b7ac93008 (table test-table [id=01eb300cae8a4ad98e723a713854c357]) successfully deleted
I20260809 15:13:24.203410 17131 raft_consensus.cc:729] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3783f8bad0a34e969f17ebcc3721eae1, State: Initialized, Role: FOLLOWER
I20260809 15:13:24.203765 17131 consensus_queue.cc:260] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } }
I20260809 15:13:24.204455 17131 ts_tablet_manager.cc:1428] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 15:13:24.205302 17156 ts_tablet_manager.cc:1918] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:13:24.205519 17156 log.cc:1196] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/20783f5981b447118e721e9b7ac93008
I20260809 15:13:24.206106 17156 ts_tablet_manager.cc:1939] T 20783f5981b447118e721e9b7ac93008 P c90942c07bde4509af9f1d7116a4a0b4: Deleting consensus metadata
I20260809 15:13:24.207993 16582 catalog_manager.cc:4841] TS c90942c07bde4509af9f1d7116a4a0b4 (127.15.11.131:46593): tablet 20783f5981b447118e721e9b7ac93008 (table test-table [id=01eb300cae8a4ad98e723a713854c357]) successfully deleted
I20260809 15:13:24.209537 17134 tablet_bootstrap.cc:492] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: No bootstrap required, opened a new log
I20260809 15:13:24.209797 17134 ts_tablet_manager.cc:1397] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.007s
I20260809 15:13:24.211061 17134 raft_consensus.cc:348] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } }
I20260809 15:13:24.211359 17134 raft_consensus.cc:374] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:24.211483 17134 raft_consensus.cc:729] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90942c07bde4509af9f1d7116a4a0b4, State: Initialized, Role: FOLLOWER
I20260809 15:13:24.211874 17134 consensus_queue.cc:260] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } }
I20260809 15:13:24.212719 17134 ts_tablet_manager.cc:1428] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 15:13:24.214339 17154 tablet_replica.cc:330] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: stopping tablet replica
I20260809 15:13:24.214830 17154 raft_consensus.cc:2229] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:24.215411 17154 raft_consensus.cc:2258] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:24.216995 17133 tablet_bootstrap.cc:492] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: No bootstrap required, opened a new log
I20260809 15:13:24.217162 17133 ts_tablet_manager.cc:1397] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.007s
I20260809 15:13:24.217679 17154 ts_tablet_manager.cc:1905] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:24.217995 17133 raft_consensus.cc:348] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } }
I20260809 15:13:24.218225 17133 raft_consensus.cc:374] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:24.218328 17133 raft_consensus.cc:729] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96094e0c3c31475197cc4f23a43dd092, State: Initialized, Role: FOLLOWER
I20260809 15:13:24.218662 17133 consensus_queue.cc:260] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } }
I20260809 15:13:24.219799 17133 ts_tablet_manager.cc:1428] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 15:13:24.224056 17154 ts_tablet_manager.cc:1918] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:13:24.224304 17154 log.cc:1196] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/20783f5981b447118e721e9b7ac93008
I20260809 15:13:24.224804 17154 ts_tablet_manager.cc:1939] T 20783f5981b447118e721e9b7ac93008 P 3783f8bad0a34e969f17ebcc3721eae1: Deleting consensus metadata
I20260809 15:13:24.226445 16582 catalog_manager.cc:4841] TS 3783f8bad0a34e969f17ebcc3721eae1 (127.15.11.130:40903): tablet 20783f5981b447118e721e9b7ac93008 (table test-table [id=01eb300cae8a4ad98e723a713854c357]) successfully deleted
I20260809 15:13:24.391379 17138 raft_consensus.cc:482] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:24.391649 17138 raft_consensus.cc:504] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } }
I20260809 15:13:24.393682 17138 leader_election.cc:288] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3783f8bad0a34e969f17ebcc3721eae1 (127.15.11.130:40903), c90942c07bde4509af9f1d7116a4a0b4 (127.15.11.131:46593)
I20260809 15:13:24.402357 17160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.394053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:40903 (local address 127.15.11.129:47691)
0809 15:13:24.394708 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:24.394728 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:24.394929 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:24.395675 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:24.395683 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:24.395718 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:24.396314 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.396330 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.397881 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.397887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.399673 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.399689 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.401521 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.401528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.401714 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.401747 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 15:13:24.401844 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 15:13:24.401997 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":343,"threads_started":1}
I20260809 15:13:24.402391 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.394058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:47691 (local address 127.15.11.130:40903)
0809 15:13:24.394279 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.394288 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.394803 (+ 515us) server_negotiation.cc:407] Connection header received
0809 15:13:24.395160 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.395169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.395293 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.395373 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:24.396518 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.397717 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.399835 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.401117 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.401279 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.401358 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.401946 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.402090 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 15:13:24.403561 17161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.394827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:46593 (local address 127.15.11.129:55921)
0809 15:13:24.397059 (+ 2232us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:24.397073 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:24.397212 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:24.397590 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:24.397594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:24.397619 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:24.398143 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.398154 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.399648 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.399653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.401085 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.401098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.402772 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.402778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.402953 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.403015 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 15:13:24.403111 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:13:24.403233 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2107,"thread_start_us":1880,"threads_started":1}
I20260809 15:13:24.403568 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.394981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:55921 (local address 127.15.11.131:46593)
0809 15:13:24.395172 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.395178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.397168 (+ 1990us) server_negotiation.cc:407] Connection header received
0809 15:13:24.397369 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.397374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.397460 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.397567 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:24.398329 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.399494 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.401247 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.402313 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.402501 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.402568 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.403141 (+ 573us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.403270 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:13:24.403498 16916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82da1572dda04affac5871d01a3edd3f" candidate_uuid: "96094e0c3c31475197cc4f23a43dd092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3783f8bad0a34e969f17ebcc3721eae1" is_pre_election: true
I20260809 15:13:24.403786 16916 raft_consensus.cc:2454] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96094e0c3c31475197cc4f23a43dd092 in term 0.
I20260809 15:13:24.404366 16693 leader_election.cc:302] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3783f8bad0a34e969f17ebcc3721eae1, 96094e0c3c31475197cc4f23a43dd092; no voters:
I20260809 15:13:24.404809 17138 raft_consensus.cc:2781] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:24.404940 17138 raft_consensus.cc:482] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:24.405066 17138 raft_consensus.cc:3037] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:24.404920 17062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82da1572dda04affac5871d01a3edd3f" candidate_uuid: "96094e0c3c31475197cc4f23a43dd092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c90942c07bde4509af9f1d7116a4a0b4" is_pre_election: true
I20260809 15:13:24.405471 17062 raft_consensus.cc:2454] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96094e0c3c31475197cc4f23a43dd092 in term 0.
I20260809 15:13:24.408329 17138 raft_consensus.cc:504] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } }
I20260809 15:13:24.409065 17138 leader_election.cc:288] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [CANDIDATE]: Term 1 election: Requested vote from peers 3783f8bad0a34e969f17ebcc3721eae1 (127.15.11.130:40903), c90942c07bde4509af9f1d7116a4a0b4 (127.15.11.131:46593)
I20260809 15:13:24.409546 17062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82da1572dda04affac5871d01a3edd3f" candidate_uuid: "96094e0c3c31475197cc4f23a43dd092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c90942c07bde4509af9f1d7116a4a0b4"
I20260809 15:13:24.409551 16916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82da1572dda04affac5871d01a3edd3f" candidate_uuid: "96094e0c3c31475197cc4f23a43dd092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3783f8bad0a34e969f17ebcc3721eae1"
I20260809 15:13:24.409763 17062 raft_consensus.cc:3037] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:24.409775 16916 raft_consensus.cc:3037] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:24.412428 17062 raft_consensus.cc:2454] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96094e0c3c31475197cc4f23a43dd092 in term 1.
I20260809 15:13:24.412868 16693 leader_election.cc:302] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96094e0c3c31475197cc4f23a43dd092, c90942c07bde4509af9f1d7116a4a0b4; no voters:
I20260809 15:13:24.412999 16916 raft_consensus.cc:2454] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96094e0c3c31475197cc4f23a43dd092 in term 1.
I20260809 15:13:24.413261 17138 raft_consensus.cc:2781] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:24.413858 17138 raft_consensus.cc:686] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 1 LEADER]: Becoming Leader. State: Replica: 96094e0c3c31475197cc4f23a43dd092, State: Running, Role: LEADER
I20260809 15:13:24.414251 17138 consensus_queue.cc:237] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } }
I20260809 15:13:24.417922 16596 catalog_manager.cc:5495] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96094e0c3c31475197cc4f23a43dd092 (127.15.11.129). New cstate: current_term: 1 leader_uuid: "96094e0c3c31475197cc4f23a43dd092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } health_report { overall_health: UNKNOWN } } }
Aug 09 15:13:24 dist-test-slave-8w9v krb5kdc[16550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288402, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:24.498975 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.483017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35157 (local address 127.0.0.1:43940)
0809 15:13:24.483780 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:24.483797 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:24.483915 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:24.484637 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:24.484642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:24.485639 (+ 997us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:24.485920 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.485934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.487058 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.487063 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.488620 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.488640 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.489485 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.489496 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.489701 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.490488 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:24.490522 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:24.493243 (+ 2721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:24.496463 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:24.496474 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:24.496483 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:24.496859 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:24.497224 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:24.497230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:24.497234 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:24.497395 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:24.497874 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:24.497882 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:24.498046 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 15:13:24.498252 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 15:13:24.498578 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":464,"threads_started":1}
I20260809 15:13:24.499046 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.483090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.15.11.190:35157)
0809 15:13:24.484327 (+ 1237us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.484331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.484349 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:13:24.484390 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.484395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.484443 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.484524 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:24.486101 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.486928 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.488786 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.489299 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.489398 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.493439 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:24.493443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:24.493446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:24.493472 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:24.496316 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.496974 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.496980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.496983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.497116 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.497494 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.497501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.497504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.497749 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:24.497907 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.498355 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.498721 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":161,"server-negotiator.queue_time_us":1147,"thread_start_us":1010,"threads_started":1}
I20260809 15:13:24.504402 16596 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:43940:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 15:13:24.505511 16596 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:24.512818 16742 tablet_service.cc:1467] Processing CreateTablet for tablet 1b0f73d87fe24d9ea247cd61d4aeae41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78c16a660ba1437cae0af75cdd7ab77f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:13:24.512974 17040 tablet_service.cc:1467] Processing CreateTablet for tablet 1b0f73d87fe24d9ea247cd61d4aeae41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78c16a660ba1437cae0af75cdd7ab77f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:13:24.513147 16890 tablet_service.cc:1467] Processing CreateTablet for tablet 1b0f73d87fe24d9ea247cd61d4aeae41 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=78c16a660ba1437cae0af75cdd7ab77f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:13:24.513491 16742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b0f73d87fe24d9ea247cd61d4aeae41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:24.513667 16890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b0f73d87fe24d9ea247cd61d4aeae41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:24.513830 17040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b0f73d87fe24d9ea247cd61d4aeae41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:24.520973 17133 tablet_bootstrap.cc:492] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: Bootstrap starting.
I20260809 15:13:24.521144 17131 tablet_bootstrap.cc:492] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: Bootstrap starting.
I20260809 15:13:24.522816 17131 tablet_bootstrap.cc:654] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:24.522835 17133 tablet_bootstrap.cc:654] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:24.523809 17134 tablet_bootstrap.cc:492] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: Bootstrap starting.
I20260809 15:13:24.525244 17134 tablet_bootstrap.cc:654] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:24.527557 17131 tablet_bootstrap.cc:492] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: No bootstrap required, opened a new log
I20260809 15:13:24.527724 17131 ts_tablet_manager.cc:1397] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260809 15:13:24.528518 17131 raft_consensus.cc:348] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } }
I20260809 15:13:24.528776 17131 raft_consensus.cc:374] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:24.528898 17131 raft_consensus.cc:729] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3783f8bad0a34e969f17ebcc3721eae1, State: Initialized, Role: FOLLOWER
I20260809 15:13:24.529172 17131 consensus_queue.cc:260] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } }
I20260809 15:13:24.530063 17134 tablet_bootstrap.cc:492] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: No bootstrap required, opened a new log
I20260809 15:13:24.530202 17133 tablet_bootstrap.cc:492] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: No bootstrap required, opened a new log
I20260809 15:13:24.530251 17134 ts_tablet_manager.cc:1397] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.001s
I20260809 15:13:24.530391 17133 ts_tablet_manager.cc:1397] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260809 15:13:24.531065 17134 raft_consensus.cc:348] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } }
I20260809 15:13:24.531301 17134 raft_consensus.cc:374] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:24.531194 17133 raft_consensus.cc:348] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } }
I20260809 15:13:24.531394 17134 raft_consensus.cc:729] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c90942c07bde4509af9f1d7116a4a0b4, State: Initialized, Role: FOLLOWER
I20260809 15:13:24.531471 17133 raft_consensus.cc:374] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:24.531553 17133 raft_consensus.cc:729] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96094e0c3c31475197cc4f23a43dd092, State: Initialized, Role: FOLLOWER
I20260809 15:13:24.531804 17133 consensus_queue.cc:260] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } }
I20260809 15:13:24.532305 17138 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.532554 17133 ts_tablet_manager.cc:1428] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 15:13:24.532598 17138 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:24.532050 17134 consensus_queue.cc:260] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } }
I20260809 15:13:24.533787 17134 ts_tablet_manager.cc:1428] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260809 15:13:24.533819 17141 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.534168 17141 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:24.534195 17135 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.534574 17135 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:24.535071 17131 ts_tablet_manager.cc:1428] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260809 15:13:24.555040 17138 raft_consensus.cc:482] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:24.555243 17138 raft_consensus.cc:504] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } }
I20260809 15:13:24.556088 17138 leader_election.cc:288] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c90942c07bde4509af9f1d7116a4a0b4 (127.15.11.131:46593), 3783f8bad0a34e969f17ebcc3721eae1 (127.15.11.130:40903)
I20260809 15:13:24.556597 17062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b0f73d87fe24d9ea247cd61d4aeae41" candidate_uuid: "96094e0c3c31475197cc4f23a43dd092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c90942c07bde4509af9f1d7116a4a0b4" is_pre_election: true
I20260809 15:13:24.556877 17062 raft_consensus.cc:2454] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96094e0c3c31475197cc4f23a43dd092 in term 0.
I20260809 15:13:24.557330 16693 leader_election.cc:302] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96094e0c3c31475197cc4f23a43dd092, c90942c07bde4509af9f1d7116a4a0b4; no voters:
I20260809 15:13:24.557447 16916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b0f73d87fe24d9ea247cd61d4aeae41" candidate_uuid: "96094e0c3c31475197cc4f23a43dd092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3783f8bad0a34e969f17ebcc3721eae1" is_pre_election: true
I20260809 15:13:24.557655 17138 raft_consensus.cc:2781] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:24.557732 16916 raft_consensus.cc:2454] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96094e0c3c31475197cc4f23a43dd092 in term 0.
I20260809 15:13:24.557757 17138 raft_consensus.cc:482] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:24.557864 17138 raft_consensus.cc:3037] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:24.559921 17138 raft_consensus.cc:504] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } }
I20260809 15:13:24.560527 17138 leader_election.cc:288] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [CANDIDATE]: Term 1 election: Requested vote from peers c90942c07bde4509af9f1d7116a4a0b4 (127.15.11.131:46593), 3783f8bad0a34e969f17ebcc3721eae1 (127.15.11.130:40903)
I20260809 15:13:24.560933 17062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b0f73d87fe24d9ea247cd61d4aeae41" candidate_uuid: "96094e0c3c31475197cc4f23a43dd092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c90942c07bde4509af9f1d7116a4a0b4"
I20260809 15:13:24.561000 16916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b0f73d87fe24d9ea247cd61d4aeae41" candidate_uuid: "96094e0c3c31475197cc4f23a43dd092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3783f8bad0a34e969f17ebcc3721eae1"
I20260809 15:13:24.561151 17062 raft_consensus.cc:3037] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:24.561221 16916 raft_consensus.cc:3037] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:24.563768 17062 raft_consensus.cc:2454] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96094e0c3c31475197cc4f23a43dd092 in term 1.
I20260809 15:13:24.564199 16693 leader_election.cc:302] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96094e0c3c31475197cc4f23a43dd092, c90942c07bde4509af9f1d7116a4a0b4; no voters:
I20260809 15:13:24.564391 16916 raft_consensus.cc:2454] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96094e0c3c31475197cc4f23a43dd092 in term 1.
I20260809 15:13:24.564476 17138 raft_consensus.cc:2781] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:24.564679 17138 raft_consensus.cc:686] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [term 1 LEADER]: Becoming Leader. State: Replica: 96094e0c3c31475197cc4f23a43dd092, State: Running, Role: LEADER
I20260809 15:13:24.565198 17138 consensus_queue.cc:237] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } }
I20260809 15:13:24.566810 17162 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: TxnStatusTablet state changed. Reason: New leader 96094e0c3c31475197cc4f23a43dd092. Latest consensus state: current_term: 1 leader_uuid: "96094e0c3c31475197cc4f23a43dd092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.567220 17162 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:13:24.568723 17176 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 15:13:24.568953 17176 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 15:13:24.568707 16596 catalog_manager.cc:5495] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96094e0c3c31475197cc4f23a43dd092 (127.15.11.129). New cstate: current_term: 1 leader_uuid: "96094e0c3c31475197cc4f23a43dd092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } health_report { overall_health: UNKNOWN } } }
I20260809 15:13:24.586906 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.580570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53522 (local address 127.15.11.129:36987)
0809 15:13:24.581268 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.581276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.581303 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:13:24.581392 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.581396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.581493 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.581658 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:24.582068 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.582884 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.584263 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.584722 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.584854 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.585496 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:24.586032 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:24.586336 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.586386 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.586466 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":749,"threads_started":1}
I20260809 15:13:24.586889 17170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.580391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:36987 (local address 127.0.0.1:53522)
0809 15:13:24.581021 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:24.581046 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:24.581188 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:24.581673 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:24.581678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:24.581701 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:13:24.581908 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.581920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.583023 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.583030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.584057 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.584068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.584885 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.584891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.585196 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.585266 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:13:24.586252 (+ 986us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:13:24.586276 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:13:24.586352 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 15:13:24.586601 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426}
I20260809 15:13:24.591461 16916 raft_consensus.cc:1261] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Refusing update from remote peer 96094e0c3c31475197cc4f23a43dd092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:24.591842 17062 raft_consensus.cc:1261] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4 [term 1 FOLLOWER]: Refusing update from remote peer 96094e0c3c31475197cc4f23a43dd092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:24.592316 17138 consensus_queue.cc:1035] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:24.593250 17138 consensus_queue.cc:1035] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:24.606149 17141 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: TxnStatusTablet state changed. Reason: New leader 96094e0c3c31475197cc4f23a43dd092. Latest consensus state: current_term: 1 leader_uuid: "96094e0c3c31475197cc4f23a43dd092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.606618 17141 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:24.608937 17138 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96094e0c3c31475197cc4f23a43dd092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.609326 17138 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:13:24.610867 17146 mvcc.cc:205] Tried to move back new op lower bound from 7316637305198800896 to 7316637305101205504. Current Snapshot: MvccSnapshot[applied={T|T < 7316637305198800896 or (T in {7316637305198800896})}]
I20260809 15:13:24.613463 17144 mvcc.cc:205] Tried to move back new op lower bound from 7316637305198800896 to 7316637305101205504. Current Snapshot: MvccSnapshot[applied={T|T < 7316637305198800896}]
I20260809 15:13:24.614689 17141 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96094e0c3c31475197cc4f23a43dd092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.614713 17135 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: TxnStatusTablet state changed. Reason: New leader 96094e0c3c31475197cc4f23a43dd092. Latest consensus state: current_term: 1 leader_uuid: "96094e0c3c31475197cc4f23a43dd092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.614993 17141 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P c90942c07bde4509af9f1d7116a4a0b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:24.615013 17135 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:24.618687 17162 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96094e0c3c31475197cc4f23a43dd092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.618940 17162 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 96094e0c3c31475197cc4f23a43dd092: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:13:24.622116 17135 tablet_replica.cc:439] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96094e0c3c31475197cc4f23a43dd092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96094e0c3c31475197cc4f23a43dd092" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36987 } } peers { permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 } } peers { permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 } } }
I20260809 15:13:24.622403 17135 tablet_replica.cc:442] T 1b0f73d87fe24d9ea247cd61d4aeae41 P 3783f8bad0a34e969f17ebcc3721eae1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:24.633953 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.618865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53530 (local address 127.15.11.129:36987)
0809 15:13:24.619067 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.619078 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.628768 (+ 9690us) server_negotiation.cc:407] Connection header received
0809 15:13:24.628981 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.628987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.629070 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.629180 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:24.629635 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.630522 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.632245 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.632662 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.632802 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.633183 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:24.633307 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:24.633403 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.633533 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.633610 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:13:24.654093 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.646708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:48693 (local address 127.15.11.129:36987)
0809 15:13:24.647143 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.647151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.647638 (+ 487us) server_negotiation.cc:407] Connection header received
0809 15:13:24.647833 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.647838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.647961 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.648056 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:24.648704 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.649794 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.651986 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.652619 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.652731 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.653288 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:24.653395 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:24.653477 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.653647 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.653740 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":287}
I20260809 15:13:24.654314 17186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.646848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:36987 (local address 127.15.11.129:48693)
0809 15:13:24.647517 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:24.647539 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:24.647699 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:24.648096 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:24.648102 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:24.648116 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:13:24.648538 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.648551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.649947 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.649952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.651813 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:24.651828 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.652821 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.652829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:24.653028 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.653099 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:13:24.653505 (+ 406us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:13:24.653519 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 15:13:24.653613 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 15:13:24.653808 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":44,"thread_start_us":315,"threads_started":1}
I20260809 15:13:24.665179 17062 raft_consensus.cc:1261] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [term 1 FOLLOWER]: Refusing update from remote peer 96094e0c3c31475197cc4f23a43dd092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:24.665349 16916 raft_consensus.cc:1261] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Refusing update from remote peer 96094e0c3c31475197cc4f23a43dd092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:24.665777 17162 consensus_queue.cc:1035] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c90942c07bde4509af9f1d7116a4a0b4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:24.666095 17138 consensus_queue.cc:1035] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3783f8bad0a34e969f17ebcc3721eae1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:24.727926 16596 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:24.728251 16596 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43924:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:24.731438 16596 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54e6f23f9be544d1b1b0497339ab1940: Sending DeleteTablet for 3 replicas of tablet 82da1572dda04affac5871d01a3edd3f
I20260809 15:13:24.732613 16890 tablet_service.cc:1514] Processing DeleteTablet for tablet 82da1572dda04affac5871d01a3edd3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:24 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:46574
I20260809 15:13:24.732695 16742 tablet_service.cc:1514] Processing DeleteTablet for tablet 82da1572dda04affac5871d01a3edd3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:24 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:53518
I20260809 15:13:24.733246 17040 tablet_service.cc:1514] Processing DeleteTablet for tablet 82da1572dda04affac5871d01a3edd3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:24 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:47086
I20260809 15:13:24.738435 17191 tablet_replica.cc:330] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: stopping tablet replica
I20260809 15:13:24.738631 17191 raft_consensus.cc:2229] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:24.738871 17191 raft_consensus.cc:2258] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:24.741036 17191 ts_tablet_manager.cc:1905] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:24.744555 17192 tablet_replica.cc:330] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: stopping tablet replica
I20260809 15:13:24.744735 17192 raft_consensus.cc:2229] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:13:24.745379 17192 raft_consensus.cc:2258] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:24.746227 17193 tablet_replica.cc:330] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: stopping tablet replica
I20260809 15:13:24.746443 17192 ts_tablet_manager.cc:1905] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:24.746698 17193 raft_consensus.cc:2229] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:24.747047 17193 raft_consensus.cc:2258] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:24.748451 17193 ts_tablet_manager.cc:1905] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:24.748488 17191 ts_tablet_manager.cc:1918] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:13:24.748605 17191 log.cc:1196] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/82da1572dda04affac5871d01a3edd3f
I20260809 15:13:24.749089 17191 ts_tablet_manager.cc:1939] T 82da1572dda04affac5871d01a3edd3f P 3783f8bad0a34e969f17ebcc3721eae1: Deleting consensus metadata
I20260809 15:13:24.750170 16582 catalog_manager.cc:4841] TS 3783f8bad0a34e969f17ebcc3721eae1 (127.15.11.130:40903): tablet 82da1572dda04affac5871d01a3edd3f (table test-table [id=045e47a9ebfa478d9f9687af7c2edbd9]) successfully deleted
I20260809 15:13:24.752321 17192 ts_tablet_manager.cc:1918] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:13:24.752470 17192 log.cc:1196] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/82da1572dda04affac5871d01a3edd3f
I20260809 15:13:24.752946 17192 ts_tablet_manager.cc:1939] T 82da1572dda04affac5871d01a3edd3f P 96094e0c3c31475197cc4f23a43dd092: Deleting consensus metadata
I20260809 15:13:24.754068 16583 catalog_manager.cc:4841] TS 96094e0c3c31475197cc4f23a43dd092 (127.15.11.129:36987): tablet 82da1572dda04affac5871d01a3edd3f (table test-table [id=045e47a9ebfa478d9f9687af7c2edbd9]) successfully deleted
I20260809 15:13:24.757956 17193 ts_tablet_manager.cc:1918] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:13:24.758185 17193 log.cc:1196] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/82da1572dda04affac5871d01a3edd3f
I20260809 15:13:24.758641 17193 ts_tablet_manager.cc:1939] T 82da1572dda04affac5871d01a3edd3f P c90942c07bde4509af9f1d7116a4a0b4: Deleting consensus metadata
I20260809 15:13:24.759549 16582 catalog_manager.cc:4841] TS c90942c07bde4509af9f1d7116a4a0b4 (127.15.11.131:46593): tablet 82da1572dda04affac5871d01a3edd3f (table test-table [id=045e47a9ebfa478d9f9687af7c2edbd9]) successfully deleted
Aug 09 15:13:24 dist-test-slave-8w9v krb5kdc[16550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288403, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:24.773175 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.754185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53540 (local address 127.15.11.129:36987)
0809 15:13:24.754412 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.754416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.755249 (+ 833us) server_negotiation.cc:407] Connection header received
0809 15:13:24.755660 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.755666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.755757 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.755883 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:24.758005 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.759216 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.760861 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.761600 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.761722 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.765711 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:24.765727 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:24.765751 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:24.765798 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:24.769995 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.770832 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.770842 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.770846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.771008 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.771471 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.771478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.771483 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.771918 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:24.772240 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.772574 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.772782 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260809 15:13:24.774086 16812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53540
I20260809 15:13:24.797592 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.781422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43942 (local address 127.15.11.190:35157)
0809 15:13:24.781653 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.781659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.782132 (+ 473us) server_negotiation.cc:407] Connection header received
0809 15:13:24.782314 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.782320 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.782406 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.782513 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:24.784162 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.785378 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.787264 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.788000 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.788158 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.791095 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:24.791102 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:24.791107 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:24.791151 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:24.794611 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.795387 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.795392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.795396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.795532 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.795946 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.795951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.795954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.796371 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:24.796620 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.797026 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.797246 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 15:13:24.798194 16596 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43942
I20260809 15:13:24.800506 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16674
I20260809 15:13:24.817797 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.802328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53542 (local address 127.15.11.129:36987)
0809 15:13:24.802529 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.802535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.802555 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:13:24.802720 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.802730 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.802804 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.802875 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:24.804571 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.805820 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.807631 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.808444 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.808571 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.811414 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:24.811420 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:24.811426 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:24.811465 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:24.815065 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.815710 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.815716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.815720 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.815875 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.816341 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.816346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.816349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.816694 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:24.816966 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.817207 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.817429 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 15:13:24.818413 16812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53542
W20260809 15:13:24.818967 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16674: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:13:24.819172 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 16674
I20260809 15:13:24.845122 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16821
Aug 09 15:13:24 dist-test-slave-8w9v krb5kdc[16550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288403, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:13:24.860419 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.845919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.15.11.130:40903)
0809 15:13:24.846112 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.846118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.846138 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:13:24.846295 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.846300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.846378 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.846480 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:24.847648 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.848880 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.850085 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.850745 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.850877 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.854975 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:24.854988 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:24.855004 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:24.855088 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:24.858016 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.858708 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.858715 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.858717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.858844 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.859158 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.859161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.859163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.859496 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:24.859720 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.859921 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.860075 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 15:13:24.861235 16958 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46588
W20260809 15:13:24.861904 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16821: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:13:24.862090 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 16821
I20260809 15:13:24.883775 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16967
Aug 09 15:13:24 dist-test-slave-8w9v krb5kdc[16550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288403, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:13:24.899626 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:24.884617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47100 (local address 127.15.11.131:46593)
0809 15:13:24.884853 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:24.884859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:24.884892 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:13:24.885079 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:24.885083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:24.885148 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:24.885254 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:24.886626 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.887348 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.888738 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:24.889177 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:24.889321 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:24.893586 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:24.893600 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:24.893618 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:24.893667 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:24.896748 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.897453 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.897461 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.897463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.897599 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:24.897972 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:24.897977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:24.897980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:24.898425 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:24.898752 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:24.899039 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 15:13:24.899263 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 15:13:24.900369 17105 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47100
W20260809 15:13:24.901002 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16967: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:13:24.901129 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 16967
I20260809 15:13:24.927459 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 16563
I20260809 15:13:24.928664 16641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:25.043196 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 16563
2026-08-09T15:13:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4626 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:25 dist-test-slave-8w9v krb5kdc[17203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:25 dist-test-slave-8w9v krb5kdc[17203](info): set up 2 sockets
Aug 09 15:13:25 dist-test-slave-8w9v krb5kdc[17203](info): commencing operation
krb5kdc: starting...
W20260809 15:13:26.156939 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:26 dist-test-slave-8w9v krb5kdc[17203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:26.353559 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:43717
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:43717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:26.698145 17216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:26.698477 17216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:26.698645 17216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:26.705915 17216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:26.706053 17216 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 15:13:26.706167 17216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:26.706286 17216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:26.706388 17216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:26.706485 17216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:26.716327 17216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:43717
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:43717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:26.717626 17216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:26.719091 17216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:26.731343 17222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:26.731370 17225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:26.731416 17223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:26.732577 17216 server_base.cc:1034] running on GCE node
I20260809 15:13:26.733378 17216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:26.734712 17216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:26.735999 17216 hybrid_clock.cc:648] HybridClock initialized: now 1786288406735912 us; error 49 us; skew 500 ppm
I20260809 15:13:26.736454 17216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:26 dist-test-slave-8w9v krb5kdc[17203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288406, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:26.739720 17216 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:26.741233 17216 webserver.cc:460] Webserver started at http://127.15.11.190:33195/ using document root <none> and password file <none>
I20260809 15:13:26.741971 17216 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:26.742120 17216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:26.742440 17216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:26.745553 17216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "e1c068ee01b04857afa59fa90b2ebd0f"
format_stamp: "Formatted at 2026-08-09 15:13:26 on dist-test-slave-8w9v"
server_key: "98eb91b525718170cc4f99a99ead5e9c"
server_key_iv: "e7753c74323abd74a8148fde5da48373"
server_key_version: "encryptionkey@0"
I20260809 15:13:26.746410 17216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "e1c068ee01b04857afa59fa90b2ebd0f"
format_stamp: "Formatted at 2026-08-09 15:13:26 on dist-test-slave-8w9v"
server_key: "98eb91b525718170cc4f99a99ead5e9c"
server_key_iv: "e7753c74323abd74a8148fde5da48373"
server_key_version: "encryptionkey@0"
I20260809 15:13:26.751554 17216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:13:26.755363 17233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:26.756728 17216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:26.756939 17216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "e1c068ee01b04857afa59fa90b2ebd0f"
format_stamp: "Formatted at 2026-08-09 15:13:26 on dist-test-slave-8w9v"
server_key: "98eb91b525718170cc4f99a99ead5e9c"
server_key_iv: "e7753c74323abd74a8148fde5da48373"
server_key_version: "encryptionkey@0"
I20260809 15:13:26.757170 17216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:26.778797 17216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:26.783017 17216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:26.783283 17216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:26.804008 17216 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:43717
I20260809 15:13:26.804014 17296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:43717 every 8 connection(s)
I20260809 15:13:26.805305 17216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:26.808887 17297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:26.811311 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 17216
I20260809 15:13:26.811532 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:26.812237 15406 external_mini_cluster.cc:1428] Setting key b2c1bb9f0f5bab5ae665b383b48774b6
I20260809 15:13:26.820813 17297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f: Bootstrap starting.
I20260809 15:13:26.824645 17297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:26.826021 17297 log.cc:824] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:13:26 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:26.828933 17297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f: No bootstrap required, opened a new log
I20260809 15:13:26.834947 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:26.814209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.15.11.190:43717)
0809 15:13:26.814967 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:26.814978 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:26.815019 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:13:26.816737 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:26.816762 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:26.817143 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:26.817463 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:26.819257 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:26.820585 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:26.822267 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:26.822953 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:26.823073 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:26.827956 (+ 4883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:26.827964 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:26.827978 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:26.828011 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:26.831031 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:26.831702 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:26.831716 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:26.831723 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:26.831888 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:26.832322 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:26.832327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:26.832332 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:26.832847 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:26.833160 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:26.833620 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 15:13:26.834025 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":360,"threads_started":1}
I20260809 15:13:26.836795 17297 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1c068ee01b04857afa59fa90b2ebd0f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43717 } }
I20260809 15:13:26.837225 17297 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:26.837335 17297 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1c068ee01b04857afa59fa90b2ebd0f, State: Initialized, Role: FOLLOWER
I20260809 15:13:26.837962 17297 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1c068ee01b04857afa59fa90b2ebd0f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43717 } }
I20260809 15:13:26.838173 17297 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:26.838265 17297 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:26.838387 17297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:26.840391 17297 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1c068ee01b04857afa59fa90b2ebd0f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43717 } }
I20260809 15:13:26.840797 17297 leader_election.cc:302] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1c068ee01b04857afa59fa90b2ebd0f; no voters:
I20260809 15:13:26.841403 17297 leader_election.cc:288] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:26.841516 17302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:26.842670 17302 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [term 1 LEADER]: Becoming Leader. State: Replica: e1c068ee01b04857afa59fa90b2ebd0f, State: Running, Role: LEADER
I20260809 15:13:26.843159 17302 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1c068ee01b04857afa59fa90b2ebd0f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43717 } }
I20260809 15:13:26.843485 17297 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:26.845647 17305 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1c068ee01b04857afa59fa90b2ebd0f. Latest consensus state: current_term: 1 leader_uuid: "e1c068ee01b04857afa59fa90b2ebd0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1c068ee01b04857afa59fa90b2ebd0f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43717 } } }
I20260809 15:13:26.845898 17304 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1c068ee01b04857afa59fa90b2ebd0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1c068ee01b04857afa59fa90b2ebd0f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43717 } } }
I20260809 15:13:26.846189 17304 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:26.846076 17305 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:26.848367 17308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:26.852928 17308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:26.861716 17308 catalog_manager.cc:1347] Generated new cluster ID: 14b767865feb4864ab649f13af3e545e
I20260809 15:13:26.861817 17308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:26.877723 17308 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:26.878978 17308 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:26.888648 17308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f: Generated new TSK 0
I20260809 15:13:26.889447 17308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:13:26.935757 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43717
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:27.260251 17325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:27.260686 17325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:27.260906 17325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:27.267854 17325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:27.267957 17325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:27.268155 17325 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 15:13:27.268229 17325 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:13:27.268368 17325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:27.277776 17325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.11.190:43717
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:27.278995 17325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:27.280366 17325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:27.293249 17330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:27.293404 17334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:27.293421 17331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:27.294401 17325 server_base.cc:1034] running on GCE node
I20260809 15:13:27.294975 17325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:27.295895 17325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:27.297107 17325 hybrid_clock.cc:648] HybridClock initialized: now 1786288407297055 us; error 66 us; skew 500 ppm
I20260809 15:13:27.297408 17325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:27 dist-test-slave-8w9v krb5kdc[17203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288407, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:27.300395 17325 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:13:27.301795 17325 webserver.cc:460] Webserver started at http://127.15.11.129:38287/ using document root <none> and password file <none>
I20260809 15:13:27.302573 17325 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:27.302716 17325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:27.303032 17325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:27.305924 17325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "9471b7da2c4d42c283f9bcd57e590432"
format_stamp: "Formatted at 2026-08-09 15:13:27 on dist-test-slave-8w9v"
server_key: "d8e9cef4c72d1c9163fd52a52a040c0e"
server_key_iv: "001c3c537661ec5e7f83024d98219580"
server_key_version: "encryptionkey@0"
I20260809 15:13:27.306656 17325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "9471b7da2c4d42c283f9bcd57e590432"
format_stamp: "Formatted at 2026-08-09 15:13:27 on dist-test-slave-8w9v"
server_key: "d8e9cef4c72d1c9163fd52a52a040c0e"
server_key_iv: "001c3c537661ec5e7f83024d98219580"
server_key_version: "encryptionkey@0"
I20260809 15:13:27.314116 17325 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260809 15:13:27.317548 17342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:27.318817 17325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:27.318986 17325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "9471b7da2c4d42c283f9bcd57e590432"
format_stamp: "Formatted at 2026-08-09 15:13:27 on dist-test-slave-8w9v"
server_key: "d8e9cef4c72d1c9163fd52a52a040c0e"
server_key_iv: "001c3c537661ec5e7f83024d98219580"
server_key_version: "encryptionkey@0"
I20260809 15:13:27.319177 17325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:27.339767 17325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:27.343624 17325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:27.343933 17325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:27.346843 17325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:27.347005 17325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:27.347153 17325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:27.347276 17325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:13:27 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288407, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:27.374747 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:27.347293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:56457 (local address 127.15.11.190:43717)
0809 15:13:27.352887 (+ 5594us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:27.352894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:27.352920 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:13:27.354107 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:27.354116 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:27.354224 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:27.354349 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:27.356891 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.358128 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.360963 (+ 2835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.361819 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.361989 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:27.367133 (+ 5144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:27.367138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:27.367141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:27.367205 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:27.370586 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:27.371582 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:27.371587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:27.371592 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:27.371799 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:27.372489 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:27.372496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:27.372501 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:27.372911 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:27.373066 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:27.374149 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0809 15:13:27.374383 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5455,"thread_start_us":5366,"threads_started":1}
I20260809 15:13:27.376170 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:27.348024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43717 (local address 127.15.11.129:56457)
0809 15:13:27.351857 (+ 3833us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:27.351891 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:27.353835 (+ 1944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:27.354631 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:27.354666 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:27.355548 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:27.356624 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:27.356660 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.358391 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.358401 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:27.360743 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:27.360769 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.362172 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.362183 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:27.362531 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:27.363350 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:27.363399 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:27.366851 (+ 3452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:27.370736 (+ 3885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:27.370744 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:27.370763 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:27.371416 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:27.371937 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:27.371949 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:27.371959 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:27.372326 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:27.373151 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:27.373162 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:27.373715 (+ 553us) client_negotiation.cc:769] Sending connection context
0809 15:13:27.374058 (+ 343us) client_negotiation.cc:240] Negotiation successful
0809 15:13:27.374571 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3620,"thread_start_us":3367,"threads_started":1}
I20260809 15:13:27.397368 17325 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:39477
I20260809 15:13:27.397418 17473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:39477 every 8 connection(s)
I20260809 15:13:27.398648 17325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:13:27.399652 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:27.385515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:35729 (local address 127.15.11.190:43717)
0809 15:13:27.385703 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:27.385710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:27.386174 (+ 464us) server_negotiation.cc:407] Connection header received
0809 15:13:27.386376 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:27.386383 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:27.386475 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:27.386583 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:27.388308 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.389329 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.390836 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.391246 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.391393 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:27.393765 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:27.393773 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:27.393780 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:27.393821 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:27.396807 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:27.397671 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:27.397675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:27.397677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:27.397807 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:27.398172 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:27.398177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:27.398182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:27.398434 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:27.398624 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:27.399082 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 15:13:27.399303 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:13:27.399857 17449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:27.385284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43717 (local address 127.15.11.129:35729)
0809 15:13:27.386069 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:27.386087 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:27.386246 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:27.386640 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:27.386649 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:27.387505 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:27.388076 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:27.388092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.389476 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.389481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:27.390696 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:27.390709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.391466 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.391475 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:27.391622 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:27.392377 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:27.392420 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:27.393489 (+ 1069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:27.397013 (+ 3524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:27.397017 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:27.397020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:27.397529 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:27.397932 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:27.397937 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:27.397941 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:27.398064 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:27.398580 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:27.398585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:27.398802 (+ 217us) client_negotiation.cc:769] Sending connection context
0809 15:13:27.398977 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 15:13:27.399333 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":527,"threads_started":1}
I20260809 15:13:27.401552 17474 heartbeater.cc:347] Connected to a master server at 127.15.11.190:43717
I20260809 15:13:27.401940 17474 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:27.402633 17474 heartbeater.cc:511] Master 127.15.11.190:43717 requested a full tablet report, sending...
I20260809 15:13:27.404637 17250 ts_manager.cc:194] Registered new tserver with Master: 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477)
I20260809 15:13:27.406070 17250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:56457
I20260809 15:13:27.407378 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 17325
I20260809 15:13:27.407588 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:13:27.408468 15406 external_mini_cluster.cc:1428] Setting key f2c3e4deed0736bb49d7788f002e2624
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:13:27.452514 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43717
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:27.779981 17478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:27.780251 17478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:27.780441 17478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:27.787341 17478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:27.787469 17478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:27.787616 17478 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 15:13:27.787705 17478 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:13:27.787845 17478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:13:27.797097 17478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.11.190:43717
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:27.798287 17478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:27.799584 17478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:27.813853 17484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:27.814067 17485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:27.814123 17487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:27.814805 17478 server_base.cc:1034] running on GCE node
I20260809 15:13:27.815447 17478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:27.816421 17478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:27.817657 17478 hybrid_clock.cc:648] HybridClock initialized: now 1786288407817589 us; error 55 us; skew 500 ppm
I20260809 15:13:27.817950 17478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:27 dist-test-slave-8w9v krb5kdc[17203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288407, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:27.821045 17478 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:13:27.822491 17478 webserver.cc:460] Webserver started at http://127.15.11.130:37085/ using document root <none> and password file <none>
I20260809 15:13:27.823133 17478 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:27.823278 17478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:27.823649 17478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:27.826529 17478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "d14d5204fe7f4f14a2fad1a9046e561c"
format_stamp: "Formatted at 2026-08-09 15:13:27 on dist-test-slave-8w9v"
server_key: "9bc7cb8e76da7fe38ca9a83c2dc76059"
server_key_iv: "3d95c411c563a3c7522f2d489797e6cc"
server_key_version: "encryptionkey@0"
I20260809 15:13:27.827276 17478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "d14d5204fe7f4f14a2fad1a9046e561c"
format_stamp: "Formatted at 2026-08-09 15:13:27 on dist-test-slave-8w9v"
server_key: "9bc7cb8e76da7fe38ca9a83c2dc76059"
server_key_iv: "3d95c411c563a3c7522f2d489797e6cc"
server_key_version: "encryptionkey@0"
I20260809 15:13:27.832410 17478 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:13:27.835731 17494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:27.837087 17478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:27.837268 17478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "d14d5204fe7f4f14a2fad1a9046e561c"
format_stamp: "Formatted at 2026-08-09 15:13:27 on dist-test-slave-8w9v"
server_key: "9bc7cb8e76da7fe38ca9a83c2dc76059"
server_key_iv: "3d95c411c563a3c7522f2d489797e6cc"
server_key_version: "encryptionkey@0"
I20260809 15:13:27.837489 17478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:27.858852 17478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:27.862865 17478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:27.863143 17478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:27.865921 17478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:27.866056 17478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:27.866196 17478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:27.866282 17478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:13:27 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288407, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:27.888518 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:27.866240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:44303 (local address 127.15.11.190:43717)
0809 15:13:27.866460 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:27.866467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:27.867831 (+ 1364us) server_negotiation.cc:407] Connection header received
0809 15:13:27.870005 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:27.870013 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:27.870137 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:27.870271 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:27.872394 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.873700 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.875246 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.876026 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.876199 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:27.881025 (+ 4826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:27.881035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:27.881042 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:27.881116 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:27.884968 (+ 3852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:27.885809 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:27.885814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:27.885818 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:27.885993 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:27.886617 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:27.886623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:27.886627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:27.886952 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:27.887123 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:27.887851 (+ 728us) server_negotiation.cc:299] Negotiation successful
0809 15:13:27.888099 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:13:27.889889 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:27.866771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43717 (local address 127.15.11.130:44303)
0809 15:13:27.867643 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:27.867686 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:27.869626 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:27.870566 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:27.870578 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:27.871336 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:27.872187 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:27.872219 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.873877 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.873880 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:27.875100 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:27.875112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.876335 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.876349 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:27.876698 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:27.877461 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:27.877504 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:27.880718 (+ 3214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:27.885116 (+ 4398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:27.885126 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:27.885146 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:27.885649 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:27.886143 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:27.886154 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:27.886158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:27.886469 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:27.887101 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:27.887109 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:27.887470 (+ 361us) client_negotiation.cc:769] Sending connection context
0809 15:13:27.887746 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 15:13:27.888293 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":354,"threads_started":1}
I20260809 15:13:27.923481 17478 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:37469
I20260809 15:13:27.923527 17626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:37469 every 8 connection(s)
I20260809 15:13:27.924919 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:27.904676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:58529 (local address 127.15.11.190:43717)
0809 15:13:27.904867 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:27.904872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:27.908740 (+ 3868us) server_negotiation.cc:407] Connection header received
0809 15:13:27.908936 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:27.908943 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:27.909048 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:27.909152 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:27.910834 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.912089 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.914680 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.915068 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.915165 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:27.918051 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:27.918059 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:27.918066 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:27.918111 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:27.921855 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:27.922845 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:27.922853 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:27.922858 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:27.923040 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:27.923517 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:27.923524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:27.923526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:27.923943 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:27.924056 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:27.924532 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 15:13:27.924698 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:13:27.925264 17478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:13:27.925384 17580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:27.904705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43717 (local address 127.15.11.130:58529)
0809 15:13:27.908625 (+ 3920us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:27.908643 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:27.908804 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:27.909223 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:27.909230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:27.910018 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:27.910631 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:27.910653 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.912321 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.912328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:27.914511 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:27.914531 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:27.915268 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:27.915275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:27.915459 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:27.916289 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:27.916326 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:27.917752 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:27.921990 (+ 4238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:27.921997 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:27.922002 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:27.922712 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:27.923175 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:27.923181 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:27.923184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:27.923374 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:27.924058 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:27.924063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:27.924290 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 15:13:27.924459 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 15:13:27.924806 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3770,"thread_start_us":3629,"threads_started":1}
I20260809 15:13:27.927606 17627 heartbeater.cc:347] Connected to a master server at 127.15.11.190:43717
I20260809 15:13:27.928457 17627 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:27.929170 17627 heartbeater.cc:511] Master 127.15.11.190:43717 requested a full tablet report, sending...
I20260809 15:13:27.930436 17250 ts_manager.cc:194] Registered new tserver with Master: d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469)
I20260809 15:13:27.931210 17250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:44303
I20260809 15:13:27.933048 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 17478
I20260809 15:13:27.933236 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:13:27.933780 15406 external_mini_cluster.cc:1428] Setting key b1ede1a45cf055c9a683821607ed4a73
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:13:27.976552 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43717
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:28.298434 17631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:28.298738 17631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:28.298925 17631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:28.306020 17631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:28.306166 17631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:28.306309 17631 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 15:13:28.306397 17631 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:13:28.306532 17631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:13:28.315748 17631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.11.190:43717
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:28.317011 17631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:28.318289 17631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:28.331447 17636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:28.331563 17639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:28.331588 17637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:28.332573 17631 server_base.cc:1034] running on GCE node
I20260809 15:13:28.333192 17631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:28.334141 17631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:28.335367 17631 hybrid_clock.cc:648] HybridClock initialized: now 1786288408335301 us; error 53 us; skew 500 ppm
I20260809 15:13:28.335692 17631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:28 dist-test-slave-8w9v krb5kdc[17203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288408, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:28.343449 17631 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:13:28.344941 17631 webserver.cc:460] Webserver started at http://127.15.11.131:41023/ using document root <none> and password file <none>
I20260809 15:13:28.345629 17631 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:28.345775 17631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:28.346122 17631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:28.349223 17631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "09e1149656794a05ab3e93dfea17390f"
format_stamp: "Formatted at 2026-08-09 15:13:28 on dist-test-slave-8w9v"
server_key: "97bc79c379e72188ee208bb0cb3d9d59"
server_key_iv: "b77a46f23d5bc127f3fb7c974037412e"
server_key_version: "encryptionkey@0"
I20260809 15:13:28.350018 17631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "09e1149656794a05ab3e93dfea17390f"
format_stamp: "Formatted at 2026-08-09 15:13:28 on dist-test-slave-8w9v"
server_key: "97bc79c379e72188ee208bb0cb3d9d59"
server_key_iv: "b77a46f23d5bc127f3fb7c974037412e"
server_key_version: "encryptionkey@0"
I20260809 15:13:28.354856 17631 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 15:13:28.358239 17647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:28.359479 17631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:13:28.359653 17631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "09e1149656794a05ab3e93dfea17390f"
format_stamp: "Formatted at 2026-08-09 15:13:28 on dist-test-slave-8w9v"
server_key: "97bc79c379e72188ee208bb0cb3d9d59"
server_key_iv: "b77a46f23d5bc127f3fb7c974037412e"
server_key_version: "encryptionkey@0"
I20260809 15:13:28.359848 17631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:28.379245 17631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:28.383257 17631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:28.383550 17631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:28.386212 17631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:28.386405 17631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 15:13:28.386569 17631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:28.386672 17631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:13:28 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288408, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:28.408756 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.386596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:47061 (local address 127.15.11.190:43717)
0809 15:13:28.386778 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:28.386783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:28.388058 (+ 1275us) server_negotiation.cc:407] Connection header received
0809 15:13:28.390364 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:28.390371 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:28.390464 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:28.390594 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:28.392951 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.394282 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.395805 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.396638 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.396798 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.401474 (+ 4676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:28.401481 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:28.401486 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:28.401528 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:28.405168 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:28.406202 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:28.406209 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:28.406215 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:28.406408 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:28.406937 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:28.406942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:28.406944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:28.407209 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:28.407396 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:28.408109 (+ 713us) server_negotiation.cc:299] Negotiation successful
0809 15:13:28.408395 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 15:13:28.409771 17474 heartbeater.cc:503] Master 127.15.11.190:43717 was elected leader, sending a full tablet report...
I20260809 15:13:28.410151 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.387024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43717 (local address 127.15.11.131:47061)
0809 15:13:28.387891 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:28.387945 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:28.390097 (+ 2152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:28.390946 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:28.390957 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:28.391747 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:28.392736 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.392765 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.394458 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.394468 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.395659 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.395670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.396891 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.396899 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.397154 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.398054 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:28.398091 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:28.401196 (+ 3105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:28.405314 (+ 4118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:28.405326 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:28.405340 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:28.406042 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:28.406541 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:28.406552 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:28.406557 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:28.406777 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:28.407346 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:28.407355 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:28.407724 (+ 369us) client_negotiation.cc:769] Sending connection context
0809 15:13:28.407997 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 15:13:28.408565 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":379,"threads_started":1}
I20260809 15:13:28.433130 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.417313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:42047 (local address 127.15.11.190:43717)
0809 15:13:28.417700 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:28.417709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:28.418108 (+ 399us) server_negotiation.cc:407] Connection header received
0809 15:13:28.418390 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:28.418396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:28.418469 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:28.418568 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:28.420071 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.421164 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.423230 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.423747 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.423882 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.426445 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:28.426455 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:28.426458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:28.426488 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:28.429650 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:28.430771 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:28.430775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:28.430777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:28.430903 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:28.431440 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:28.431449 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:28.431455 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:28.431868 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:28.432053 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:28.432546 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 15:13:28.432767 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260809 15:13:28.433279 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.417317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43717 (local address 127.15.11.131:42047)
0809 15:13:28.417977 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:28.418006 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:28.418184 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:28.418600 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:28.418604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:28.419295 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:28.419824 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.419842 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.421305 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.421310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.423035 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.423068 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.424196 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.424211 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.424460 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.425194 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:28.425245 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:28.426242 (+ 997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:28.429842 (+ 3600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:28.429850 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:28.429857 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:28.430639 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:28.431035 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:28.431044 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:28.431047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:28.431284 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:28.432029 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:28.432035 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:28.432278 (+ 243us) client_negotiation.cc:769] Sending connection context
0809 15:13:28.432448 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 15:13:28.432784 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":397,"threads_started":1}
I20260809 15:13:28.442227 17631 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:43567
I20260809 15:13:28.442313 17781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:43567 every 8 connection(s)
I20260809 15:13:28.443526 17631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:13:28.444597 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 17631
I20260809 15:13:28.444809 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:13:28.445538 15406 external_mini_cluster.cc:1428] Setting key bd9653e953cd0ba2c40aa19ae117b773
I20260809 15:13:28.445601 17782 heartbeater.cc:347] Connected to a master server at 127.15.11.190:43717
I20260809 15:13:28.445789 17782 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:28.446460 17782 heartbeater.cc:511] Master 127.15.11.190:43717 requested a full tablet report, sending...
I20260809 15:13:28.448344 17251 ts_manager.cc:194] Registered new tserver with Master: 09e1149656794a05ab3e93dfea17390f (127.15.11.131:43567)
I20260809 15:13:28.449344 17251 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:47061
I20260809 15:13:28.461107 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:13:28 dist-test-slave-8w9v krb5kdc[17203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:13:28 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:28.511186 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.498120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.15.11.190:43717)
0809 15:13:28.498309 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:28.498314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:28.498642 (+ 328us) server_negotiation.cc:407] Connection header received
0809 15:13:28.498796 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:28.498800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:28.498876 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:28.498976 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:28.500378 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.501087 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.502226 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.502628 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.502729 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.506487 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:28.506493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:28.506496 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:28.506551 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:28.509087 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:28.509635 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:28.509639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:28.509641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:28.509758 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:28.510073 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:28.510076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:28.510077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:28.510349 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:28.510526 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:28.510754 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 15:13:28.510906 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:13:28.515558 17251 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:13:28.522125 17251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:28.545409 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.534792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.15.11.130:37469)
0809 15:13:28.536547 (+ 1755us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:28.536555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:28.536578 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:13:28.536673 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:28.536680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:28.536917 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:28.538030 (+ 1113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:28.538256 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.539718 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.542593 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.544019 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.544206 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.544425 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:28.544874 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 15:13:28.545072 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1584,"thread_start_us":1432,"threads_started":1}
I20260809 15:13:28.545496 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.534119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:37469 (local address 127.0.0.1:54702)
0809 15:13:28.535562 (+ 1443us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:28.535582 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:28.535697 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:28.537155 (+ 1458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:28.537161 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:28.537194 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:28.537806 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.537822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.539862 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.539868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.542442 (+ 2574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.542455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.544282 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.544290 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.544549 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.544623 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 15:13:28.544754 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 15:13:28.544935 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1310,"mutex_wait_us":34,"thread_start_us":1861,"threads_started":1}
I20260809 15:13:28.545644 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.533766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:43567 (local address 127.0.0.1:44400)
0809 15:13:28.534616 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:28.534664 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:28.534815 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:28.536563 (+ 1748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:28.536570 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:28.536602 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:28.537004 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.537025 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.539961 (+ 2936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.539967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.542707 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.542724 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.544647 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.544654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.544870 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.544969 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 15:13:28.545051 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 15:13:28.545175 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":259,"thread_start_us":447,"threads_started":1}
I20260809 15:13:28.546306 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.535709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:39477 (local address 127.0.0.1:42544)
0809 15:13:28.537481 (+ 1772us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:28.537495 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:28.537647 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:28.538612 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:28.538617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:28.538648 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:28.538933 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.538940 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.540405 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.540411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.542761 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.542775 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.544628 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.544634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.544924 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.544975 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 15:13:28.545056 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:13:28.545184 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1636,"mutex_wait_us":121,"thread_start_us":1441,"threads_started":1}
I20260809 15:13:28.546597 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.534115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.15.11.131:43567)
0809 15:13:28.535861 (+ 1746us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:28.535870 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:28.535901 (+ 31us) server_negotiation.cc:407] Connection header received
0809 15:13:28.536021 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:28.536030 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:28.536324 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:28.537268 (+ 944us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:28.537664 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.539757 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.542903 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.544331 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.544537 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.544763 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:28.546118 (+ 1355us) server_negotiation.cc:299] Negotiation successful
0809 15:13:28.546285 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1552,"thread_start_us":1380,"threads_started":1}
I20260809 15:13:28.546998 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.536265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42544 (local address 127.15.11.129:39477)
0809 15:13:28.536949 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:28.536960 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:28.537854 (+ 894us) server_negotiation.cc:407] Connection header received
0809 15:13:28.537965 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:28.537971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:28.538190 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:28.538381 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:28.539122 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.540224 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.542945 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.544064 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.544282 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.544874 (+ 592us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:28.546573 (+ 1699us) server_negotiation.cc:299] Negotiation successful
0809 15:13:28.546768 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":384,"threads_started":1}
I20260809 15:13:28.549566 17550 tablet_service.cc:1467] Processing CreateTablet for tablet f93148cc8e344f3783a6393efc930a5d (DEFAULT_TABLE table=test-table [id=e302d72800a34f039ef81280625000a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:28.550108 17702 tablet_service.cc:1467] Processing CreateTablet for tablet f93148cc8e344f3783a6393efc930a5d (DEFAULT_TABLE table=test-table [id=e302d72800a34f039ef81280625000a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:28.551244 17550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f93148cc8e344f3783a6393efc930a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:28.550871 17396 tablet_service.cc:1467] Processing CreateTablet for tablet f93148cc8e344f3783a6393efc930a5d (DEFAULT_TABLE table=test-table [id=e302d72800a34f039ef81280625000a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:28.551510 17702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f93148cc8e344f3783a6393efc930a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:28.552170 17396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f93148cc8e344f3783a6393efc930a5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:28.562378 17804 tablet_bootstrap.cc:492] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: Bootstrap starting.
I20260809 15:13:28.565948 17805 tablet_bootstrap.cc:492] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: Bootstrap starting.
I20260809 15:13:28.566622 17804 tablet_bootstrap.cc:654] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:28.567606 17807 tablet_bootstrap.cc:492] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: Bootstrap starting.
I20260809 15:13:28.568365 17804 log.cc:824] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:28.569424 17805 tablet_bootstrap.cc:654] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:28.570688 17807 tablet_bootstrap.cc:654] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:28.570698 17805 log.cc:824] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:28.571538 17804 tablet_bootstrap.cc:492] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: No bootstrap required, opened a new log
I20260809 15:13:28.571839 17804 ts_tablet_manager.cc:1397] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260809 15:13:28.571873 17807 log.cc:824] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:28.573798 17805 tablet_bootstrap.cc:492] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: No bootstrap required, opened a new log
I20260809 15:13:28.574086 17805 ts_tablet_manager.cc:1397] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260809 15:13:28.578063 17807 tablet_bootstrap.cc:492] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: No bootstrap required, opened a new log
I20260809 15:13:28.578312 17807 ts_tablet_manager.cc:1397] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260809 15:13:28.582309 17804 raft_consensus.cc:348] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } }
I20260809 15:13:28.582850 17804 raft_consensus.cc:374] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:28.582988 17804 raft_consensus.cc:729] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14d5204fe7f4f14a2fad1a9046e561c, State: Initialized, Role: FOLLOWER
I20260809 15:13:28.583767 17804 consensus_queue.cc:260] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } }
I20260809 15:13:28.584458 17805 raft_consensus.cc:348] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } }
I20260809 15:13:28.584877 17627 heartbeater.cc:503] Master 127.15.11.190:43717 was elected leader, sending a full tablet report...
I20260809 15:13:28.584925 17805 raft_consensus.cc:374] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:28.585119 17805 raft_consensus.cc:729] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9471b7da2c4d42c283f9bcd57e590432, State: Initialized, Role: FOLLOWER
I20260809 15:13:28.585608 17804 ts_tablet_manager.cc:1428] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260809 15:13:28.585861 17805 consensus_queue.cc:260] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } }
I20260809 15:13:28.586396 17807 raft_consensus.cc:348] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } }
I20260809 15:13:28.586853 17807 raft_consensus.cc:374] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:28.586988 17807 raft_consensus.cc:729] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09e1149656794a05ab3e93dfea17390f, State: Initialized, Role: FOLLOWER
I20260809 15:13:28.587666 17805 ts_tablet_manager.cc:1428] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: Time spent starting tablet: real 0.013s user 0.011s sys 0.003s
I20260809 15:13:28.587567 17807 consensus_queue.cc:260] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } }
I20260809 15:13:28.588779 17782 heartbeater.cc:503] Master 127.15.11.190:43717 was elected leader, sending a full tablet report...
I20260809 15:13:28.590544 17807 ts_tablet_manager.cc:1428] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
W20260809 15:13:28.652611 17475 tablet.cc:2369] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:13:28.678082 17628 tablet.cc:2369] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:13:28.697386 17783 tablet.cc:2369] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:28.902460 17814 raft_consensus.cc:482] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:28.902789 17814 raft_consensus.cc:504] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } }
I20260809 15:13:28.904596 17814 leader_election.cc:288] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469), 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477)
I20260809 15:13:28.912150 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.905224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:56129 (local address 127.15.11.130:37469)
0809 15:13:28.905415 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:28.905423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:28.905648 (+ 225us) server_negotiation.cc:407] Connection header received
0809 15:13:28.905747 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:28.905754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:28.905833 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:28.905925 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:28.906550 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.907558 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.909923 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.911139 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.911325 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.911395 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:28.911751 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 15:13:28.911877 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:13:28.912182 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.905333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:57685 (local address 127.15.11.129:39477)
0809 15:13:28.905522 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:28.905526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:28.905542 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:13:28.905636 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:28.905640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:28.905720 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:28.905821 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:28.906341 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.907376 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.909923 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.911138 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.911316 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.911404 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:28.911750 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 15:13:28.911877 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:13:28.912328 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.904929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:39477 (local address 127.15.11.131:57685)
0809 15:13:28.905128 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:28.905149 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:28.905285 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:28.905863 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:28.905867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:28.905908 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:28.906183 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.906195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.907533 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.907537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.909763 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.909775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.911320 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.911327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.911590 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.911625 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 15:13:28.911712 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 15:13:28.911942 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":176}
I20260809 15:13:28.912390 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:28.904953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:37469 (local address 127.15.11.131:56129)
0809 15:13:28.905551 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:28.905564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:28.905655 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:28.905932 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:28.905936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:28.905971 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:28.906415 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.906424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.907676 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.907680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.909760 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:28.909775 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:28.911366 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:28.911372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:28.911562 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:28.911612 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 15:13:28.911712 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 15:13:28.911942 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":103,"thread_start_us":734,"threads_started":1}
I20260809 15:13:28.913605 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93148cc8e344f3783a6393efc930a5d" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9471b7da2c4d42c283f9bcd57e590432" is_pre_election: true
I20260809 15:13:28.913605 17577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93148cc8e344f3783a6393efc930a5d" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" is_pre_election: true
I20260809 15:13:28.914207 17419 raft_consensus.cc:2454] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 0.
I20260809 15:13:28.914207 17577 raft_consensus.cc:2454] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 0.
I20260809 15:13:28.914975 17651 leader_election.cc:302] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09e1149656794a05ab3e93dfea17390f, 9471b7da2c4d42c283f9bcd57e590432, d14d5204fe7f4f14a2fad1a9046e561c; no voters:
I20260809 15:13:28.915439 17814 raft_consensus.cc:2781] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:28.915608 17814 raft_consensus.cc:482] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:28.915717 17814 raft_consensus.cc:3037] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:28.918011 17814 raft_consensus.cc:504] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } }
I20260809 15:13:28.918658 17814 leader_election.cc:288] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 election: Requested vote from peers d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469), 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477)
I20260809 15:13:28.919102 17577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93148cc8e344f3783a6393efc930a5d" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14d5204fe7f4f14a2fad1a9046e561c"
I20260809 15:13:28.919134 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f93148cc8e344f3783a6393efc930a5d" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9471b7da2c4d42c283f9bcd57e590432"
I20260809 15:13:28.919291 17577 raft_consensus.cc:3037] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:28.919353 17419 raft_consensus.cc:3037] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:28.921495 17577 raft_consensus.cc:2454] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 1.
I20260809 15:13:28.921542 17419 raft_consensus.cc:2454] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 1.
I20260809 15:13:28.921939 17652 leader_election.cc:302] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e1149656794a05ab3e93dfea17390f, d14d5204fe7f4f14a2fad1a9046e561c; no voters:
I20260809 15:13:28.922283 17814 raft_consensus.cc:2781] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:28.922803 17814 raft_consensus.cc:686] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 1 LEADER]: Becoming Leader. State: Replica: 09e1149656794a05ab3e93dfea17390f, State: Running, Role: LEADER
I20260809 15:13:28.923357 17814 consensus_queue.cc:237] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } }
I20260809 15:13:28.928763 17249 catalog_manager.cc:5495] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f reported cstate change: term changed from 0 to 1, leader changed from <none> to 09e1149656794a05ab3e93dfea17390f (127.15.11.131). New cstate: current_term: 1 leader_uuid: "09e1149656794a05ab3e93dfea17390f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } health_report { overall_health: UNKNOWN } } }
Aug 09 15:13:29 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288406, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:29.027861 17823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.014596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43717 (local address 127.0.0.1:35042)
0809 15:13:29.015359 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:29.015388 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:29.015514 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:29.015859 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:29.015863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:29.016637 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:29.016889 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.016898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.017990 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.017993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.018920 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.018931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.019572 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.019576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.019698 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.020358 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:29.020382 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:29.022631 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:29.025680 (+ 3049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:29.025706 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:29.025712 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:29.026113 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:29.026427 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:29.026430 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:29.026432 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:29.026567 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:29.027031 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:29.027037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:29.027182 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 15:13:29.027313 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 15:13:29.027534 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":405,"threads_started":1}
I20260809 15:13:29.027989 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.014640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35042 (local address 127.15.11.190:43717)
0809 15:13:29.015427 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:29.015431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:29.015454 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:13:29.015641 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:29.015644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:29.015702 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:29.015812 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:29.017049 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.017870 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.019040 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.019418 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.019505 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.022862 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:29.022868 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:29.022871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:29.022909 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:29.025566 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:29.026215 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:29.026219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:29.026221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:29.026335 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:29.026654 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:29.026656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:29.026658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:29.026908 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:29.027032 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:29.027394 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 15:13:29.027670 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":653,"thread_start_us":479,"threads_started":1}
I20260809 15:13:29.033005 17249 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:35042:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 15:13:29.034096 17249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:29.040664 17396 tablet_service.cc:1467] Processing CreateTablet for tablet d5fcbc7075bc4c8b8a6bdee94ecb46a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fe312f62f044492a961f1736c0f41427]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:13:29.040978 17702 tablet_service.cc:1467] Processing CreateTablet for tablet d5fcbc7075bc4c8b8a6bdee94ecb46a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fe312f62f044492a961f1736c0f41427]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:13:29.041008 17550 tablet_service.cc:1467] Processing CreateTablet for tablet d5fcbc7075bc4c8b8a6bdee94ecb46a2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fe312f62f044492a961f1736c0f41427]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:13:29.041754 17396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fcbc7075bc4c8b8a6bdee94ecb46a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:29.041868 17702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fcbc7075bc4c8b8a6bdee94ecb46a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:29.041868 17550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5fcbc7075bc4c8b8a6bdee94ecb46a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:29.047420 17805 tablet_bootstrap.cc:492] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: Bootstrap starting.
I20260809 15:13:29.048897 17805 tablet_bootstrap.cc:654] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:29.049649 17804 tablet_bootstrap.cc:492] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: Bootstrap starting.
I20260809 15:13:29.049997 17807 tablet_bootstrap.cc:492] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: Bootstrap starting.
I20260809 15:13:29.051340 17807 tablet_bootstrap.cc:654] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:29.051807 17804 tablet_bootstrap.cc:654] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:29.054008 17805 tablet_bootstrap.cc:492] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: No bootstrap required, opened a new log
I20260809 15:13:29.054091 17807 tablet_bootstrap.cc:492] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: No bootstrap required, opened a new log
I20260809 15:13:29.054168 17805 ts_tablet_manager.cc:1397] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:13:29.054229 17807 ts_tablet_manager.cc:1397] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.002s
I20260809 15:13:29.054546 17804 tablet_bootstrap.cc:492] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: No bootstrap required, opened a new log
I20260809 15:13:29.054718 17804 ts_tablet_manager.cc:1397] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 15:13:29.054960 17805 raft_consensus.cc:348] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.054957 17807 raft_consensus.cc:348] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.055212 17807 raft_consensus.cc:374] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:29.055244 17805 raft_consensus.cc:374] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:29.055333 17807 raft_consensus.cc:729] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09e1149656794a05ab3e93dfea17390f, State: Initialized, Role: FOLLOWER
I20260809 15:13:29.055338 17805 raft_consensus.cc:729] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9471b7da2c4d42c283f9bcd57e590432, State: Initialized, Role: FOLLOWER
I20260809 15:13:29.055562 17807 consensus_queue.cc:260] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.055562 17805 consensus_queue.cc:260] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.055713 17804 raft_consensus.cc:348] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.056007 17804 raft_consensus.cc:374] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:29.056195 17807 ts_tablet_manager.cc:1428] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 15:13:29.056170 17804 raft_consensus.cc:729] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14d5204fe7f4f14a2fad1a9046e561c, State: Initialized, Role: FOLLOWER
I20260809 15:13:29.056015 17813 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.056440 17813 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:29.056432 17804 consensus_queue.cc:260] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.056551 17814 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.056844 17814 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:29.057417 17805 ts_tablet_manager.cc:1428] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:13:29.057204 17811 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.057567 17811 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:29.057725 17804 ts_tablet_manager.cc:1428] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 15:13:29.095466 17811 raft_consensus.cc:482] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:29.095671 17811 raft_consensus.cc:504] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.097173 17811 leader_election.cc:288] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477), 09e1149656794a05ab3e93dfea17390f (127.15.11.131:43567)
I20260809 15:13:29.105996 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.097614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:51305 (local address 127.15.11.129:39477)
0809 15:13:29.097842 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:29.097851 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:29.098588 (+ 737us) server_negotiation.cc:407] Connection header received
0809 15:13:29.098815 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:29.098822 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:29.098918 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:29.099011 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:29.099586 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.100747 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.103514 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.104788 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.104952 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.105031 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:29.105582 (+ 551us) server_negotiation.cc:299] Negotiation successful
0809 15:13:29.105719 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:13:29.106032 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.097617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:52477 (local address 127.15.11.131:43567)
0809 15:13:29.098316 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:29.098325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:29.098923 (+ 598us) server_negotiation.cc:407] Connection header received
0809 15:13:29.099086 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:29.099090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:29.099147 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:29.099248 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:29.099902 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.100959 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.103514 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.104786 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.104968 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.105381 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:29.105582 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 15:13:29.105722 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":383,"threads_started":1}
I20260809 15:13:29.106272 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.097443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:39477 (local address 127.15.11.130:51305)
0809 15:13:29.098478 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:29.098498 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:29.098669 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:29.099032 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:29.099037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:29.099065 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:29.099422 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.099433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.100877 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.100881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.103349 (+ 2468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.103364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.105104 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.105109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.105358 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.105438 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 15:13:29.105539 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 15:13:29.105733 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":153,"thread_start_us":758,"threads_started":1}
I20260809 15:13:29.106295 17829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.097443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:43567 (local address 127.15.11.130:52477)
0809 15:13:29.098808 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:29.098822 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:29.098936 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:29.099284 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:29.099289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:29.099314 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:29.099727 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.099737 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.101083 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.101088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.103349 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.103364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.105086 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.105093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.105362 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.105447 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 15:13:29.105539 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 15:13:29.105733 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"mutex_wait_us":215,"thread_start_us":1000,"threads_started":1}
I20260809 15:13:29.107048 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fcbc7075bc4c8b8a6bdee94ecb46a2" candidate_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9471b7da2c4d42c283f9bcd57e590432" is_pre_election: true
I20260809 15:13:29.107139 17730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fcbc7075bc4c8b8a6bdee94ecb46a2" candidate_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09e1149656794a05ab3e93dfea17390f" is_pre_election: true
I20260809 15:13:29.107311 17419 raft_consensus.cc:2454] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d14d5204fe7f4f14a2fad1a9046e561c in term 0.
I20260809 15:13:29.107559 17730 raft_consensus.cc:2454] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d14d5204fe7f4f14a2fad1a9046e561c in term 0.
I20260809 15:13:29.107803 17497 leader_election.cc:302] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9471b7da2c4d42c283f9bcd57e590432, d14d5204fe7f4f14a2fad1a9046e561c; no voters:
I20260809 15:13:29.108243 17811 raft_consensus.cc:2781] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:29.108376 17811 raft_consensus.cc:482] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:29.108489 17811 raft_consensus.cc:3037] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:29.110618 17811 raft_consensus.cc:504] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.111222 17811 leader_election.cc:288] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [CANDIDATE]: Term 1 election: Requested vote from peers 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477), 09e1149656794a05ab3e93dfea17390f (127.15.11.131:43567)
I20260809 15:13:29.111621 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fcbc7075bc4c8b8a6bdee94ecb46a2" candidate_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9471b7da2c4d42c283f9bcd57e590432"
I20260809 15:13:29.111696 17730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5fcbc7075bc4c8b8a6bdee94ecb46a2" candidate_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09e1149656794a05ab3e93dfea17390f"
I20260809 15:13:29.111862 17419 raft_consensus.cc:3037] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:29.111919 17730 raft_consensus.cc:3037] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:29.114275 17419 raft_consensus.cc:2454] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d14d5204fe7f4f14a2fad1a9046e561c in term 1.
I20260809 15:13:29.114580 17730 raft_consensus.cc:2454] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d14d5204fe7f4f14a2fad1a9046e561c in term 1.
I20260809 15:13:29.114639 17497 leader_election.cc:302] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9471b7da2c4d42c283f9bcd57e590432, d14d5204fe7f4f14a2fad1a9046e561c; no voters:
I20260809 15:13:29.114930 17811 raft_consensus.cc:2781] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:29.115374 17811 raft_consensus.cc:686] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 LEADER]: Becoming Leader. State: Replica: d14d5204fe7f4f14a2fad1a9046e561c, State: Running, Role: LEADER
I20260809 15:13:29.115919 17811 consensus_queue.cc:237] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.119251 17831 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: TxnStatusTablet state changed. Reason: New leader d14d5204fe7f4f14a2fad1a9046e561c. Latest consensus state: current_term: 1 leader_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.119774 17831 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:13:29.120631 17833 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 15:13:29.120555 17251 catalog_manager.cc:5495] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c reported cstate change: term changed from 0 to 1, leader changed from <none> to d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130). New cstate: current_term: 1 leader_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } health_report { overall_health: UNKNOWN } } }
I20260809 15:13:29.120905 17833 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 15:13:29.142791 17823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.136675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:37469 (local address 127.0.0.1:54714)
0809 15:13:29.136904 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:29.136922 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:29.137038 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:29.137543 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:29.137546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:29.137555 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:13:29.137719 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.137726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.139233 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.139236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.140259 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.140272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.141251 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.141259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.141393 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.141447 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:13:29.142397 (+ 950us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:13:29.142414 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 15:13:29.142467 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 15:13:29.142579 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 15:13:29.142889 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.136743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.15.11.130:37469)
0809 15:13:29.136927 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:29.136933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:29.136997 (+ 64us) server_negotiation.cc:407] Connection header received
0809 15:13:29.137245 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:29.137251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:29.137364 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:29.137524 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:29.137904 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.139117 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.140399 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.141079 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.141232 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.141650 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:29.142294 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:29.142441 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:29.142500 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 15:13:29.142576 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:13:29.150597 17419 raft_consensus.cc:1261] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Refusing update from remote peer d14d5204fe7f4f14a2fad1a9046e561c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:29.150758 17730 raft_consensus.cc:1261] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Refusing update from remote peer d14d5204fe7f4f14a2fad1a9046e561c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:29.151633 17811 consensus_queue.cc:1035] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:29.152021 17831 consensus_queue.cc:1035] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c [LEADER]: Connected to new peer: Peer: permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:29.159677 17814 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: TxnStatusTablet state changed. Reason: New leader d14d5204fe7f4f14a2fad1a9046e561c. Latest consensus state: current_term: 1 leader_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.160089 17814 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:29.177970 17836 mvcc.cc:205] Tried to move back new op lower bound from 7316637323871264768 to 7316637323742785536. Current Snapshot: MvccSnapshot[applied={T|T < 7316637323871264768}]
I20260809 15:13:29.180697 17831 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.181046 17831 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:13:29.190090 17811 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.190454 17811 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P d14d5204fe7f4f14a2fad1a9046e561c: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:13:29.194522 17813 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: TxnStatusTablet state changed. Reason: New leader d14d5204fe7f4f14a2fad1a9046e561c. Latest consensus state: current_term: 1 leader_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.195200 17813 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:29.206154 17814 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.206450 17814 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 09e1149656794a05ab3e93dfea17390f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:29.217789 17813 tablet_replica.cc:439] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } } }
I20260809 15:13:29.218055 17813 tablet_replica.cc:442] T d5fcbc7075bc4c8b8a6bdee94ecb46a2 P 9471b7da2c4d42c283f9bcd57e590432: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:13:29.221151 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.208294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.15.11.131:43567)
0809 15:13:29.208487 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:29.208493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:29.213150 (+ 4657us) server_negotiation.cc:407] Connection header received
0809 15:13:29.213399 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:29.213409 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:29.213533 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:29.213695 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:29.214569 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.215982 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.218395 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.218958 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.219058 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.219707 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:29.220391 (+ 684us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:29.220507 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:29.220772 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 15:13:29.220858 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:13:29.237186 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.231779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:37733 (local address 127.15.11.130:37469)
0809 15:13:29.231944 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:29.231948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:29.232547 (+ 599us) server_negotiation.cc:407] Connection header received
0809 15:13:29.232742 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:29.232746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:29.232810 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:29.232914 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:29.233374 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.234066 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.235359 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.235803 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.235957 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.236468 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:29.236568 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:29.236666 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:29.236804 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 15:13:29.236889 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 15:13:29.237282 17851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.231832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:37469 (local address 127.15.11.131:37733)
0809 15:13:29.232444 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:29.232459 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:29.232602 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:29.232929 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:29.232933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:29.232943 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:13:29.233209 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.233217 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.234199 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.234206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.235224 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.235235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.236006 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.236013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.236211 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.236263 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:13:29.236668 (+ 405us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:13:29.236690 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:13:29.236769 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 15:13:29.236898 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":338,"threads_started":1}
I20260809 15:13:29.247272 17419 raft_consensus.cc:1261] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Refusing update from remote peer 09e1149656794a05ab3e93dfea17390f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:29.247761 17577 raft_consensus.cc:1261] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Refusing update from remote peer 09e1149656794a05ab3e93dfea17390f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:29.247970 17816 consensus_queue.cc:1035] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:29.249061 17816 consensus_queue.cc:1035] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:29.254518 17837 mvcc.cc:205] Tried to move back new op lower bound from 7316637324271116288 to 7316637322954903552. Current Snapshot: MvccSnapshot[applied={T|T < 7316637324271116288}]
I20260809 15:13:29.287755 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.280030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:48121 (local address 127.15.11.131:43567)
0809 15:13:29.280231 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:29.280257 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:29.281134 (+ 877us) server_negotiation.cc:407] Connection header received
0809 15:13:29.281349 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:29.281357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:29.281461 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:29.281561 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:29.282255 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.283458 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.285711 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.286337 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.286444 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.287099 (+ 655us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:29.287210 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:29.287290 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:29.287468 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 15:13:29.287530 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:13:29.288003 17858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:29.280093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:43567 (local address 127.15.11.130:48121)
0809 15:13:29.281032 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:29.281048 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:29.281217 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:29.281598 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:29.281603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:29.281617 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:13:29.282069 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.282082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.283645 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.283650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.285543 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:29.285558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:29.286638 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:29.286644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:29.286820 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:29.286891 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:13:29.287327 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:13:29.287339 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 15:13:29.287440 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 15:13:29.287576 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":502,"threads_started":1}
I20260809 15:13:29.326123 17250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:29.326433 17250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:29.330547 17250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f: Sending DeleteTablet for 3 replicas of tablet f93148cc8e344f3783a6393efc930a5d
I20260809 15:13:29.331943 17702 tablet_service.cc:1514] Processing DeleteTablet for tablet f93148cc8e344f3783a6393efc930a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:29 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:44400
I20260809 15:13:29.331959 17550 tablet_service.cc:1514] Processing DeleteTablet for tablet f93148cc8e344f3783a6393efc930a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:29 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:54702
I20260809 15:13:29.333420 17396 tablet_service.cc:1514] Processing DeleteTablet for tablet f93148cc8e344f3783a6393efc930a5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:29 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:42544
I20260809 15:13:29.339109 17251 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:13:29.339722 17251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:29.347981 17701 tablet_service.cc:1467] Processing CreateTablet for tablet 7534018f116f43a39223e96f157b7a03 (DEFAULT_TABLE table=test-table [id=1070f44add594410a821de064ac587c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:29.348380 17549 tablet_service.cc:1467] Processing CreateTablet for tablet 7534018f116f43a39223e96f157b7a03 (DEFAULT_TABLE table=test-table [id=1070f44add594410a821de064ac587c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:29.348743 17701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7534018f116f43a39223e96f157b7a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:29.348990 17549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7534018f116f43a39223e96f157b7a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:29.351758 17861 tablet_replica.cc:330] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: stopping tablet replica
I20260809 15:13:29.352293 17861 raft_consensus.cc:2229] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:29.352664 17861 raft_consensus.cc:2258] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:29.352982 17395 tablet_service.cc:1467] Processing CreateTablet for tablet 7534018f116f43a39223e96f157b7a03 (DEFAULT_TABLE table=test-table [id=1070f44add594410a821de064ac587c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:29.353535 17395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7534018f116f43a39223e96f157b7a03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:29.354709 17861 ts_tablet_manager.cc:1905] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:29.362226 17861 ts_tablet_manager.cc:1918] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:13:29.362443 17861 log.cc:1196] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/f93148cc8e344f3783a6393efc930a5d
I20260809 15:13:29.363095 17861 ts_tablet_manager.cc:1939] T f93148cc8e344f3783a6393efc930a5d P d14d5204fe7f4f14a2fad1a9046e561c: Deleting consensus metadata
I20260809 15:13:29.364243 17807 tablet_bootstrap.cc:492] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: Bootstrap starting.
I20260809 15:13:29.364668 17805 tablet_bootstrap.cc:492] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: Bootstrap starting.
I20260809 15:13:29.365096 17237 catalog_manager.cc:4841] TS d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469): tablet f93148cc8e344f3783a6393efc930a5d (table test-table [id=e302d72800a34f039ef81280625000a3]) successfully deleted
I20260809 15:13:29.366155 17807 tablet_bootstrap.cc:654] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:29.366184 17805 tablet_bootstrap.cc:654] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:29.368419 17805 tablet_bootstrap.cc:492] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: No bootstrap required, opened a new log
I20260809 15:13:29.368580 17805 ts_tablet_manager.cc:1397] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260809 15:13:29.369318 17805 raft_consensus.cc:348] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.369580 17805 raft_consensus.cc:374] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:29.369673 17805 raft_consensus.cc:729] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9471b7da2c4d42c283f9bcd57e590432, State: Initialized, Role: FOLLOWER
I20260809 15:13:29.369908 17805 consensus_queue.cc:260] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.370533 17805 ts_tablet_manager.cc:1428] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 15:13:29.374020 17807 tablet_bootstrap.cc:492] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: No bootstrap required, opened a new log
I20260809 15:13:29.374259 17807 ts_tablet_manager.cc:1397] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.005s
I20260809 15:13:29.375330 17807 raft_consensus.cc:348] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.375612 17807 raft_consensus.cc:374] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:29.375710 17807 raft_consensus.cc:729] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09e1149656794a05ab3e93dfea17390f, State: Initialized, Role: FOLLOWER
I20260809 15:13:29.375949 17807 consensus_queue.cc:260] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.377990 17807 ts_tablet_manager.cc:1428] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 15:13:29.383500 17804 tablet_bootstrap.cc:492] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: Bootstrap starting.
I20260809 15:13:29.385321 17804 tablet_bootstrap.cc:654] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:29.387696 17804 tablet_bootstrap.cc:492] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: No bootstrap required, opened a new log
I20260809 15:13:29.387868 17804 ts_tablet_manager.cc:1397] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 15:13:29.388690 17804 raft_consensus.cc:348] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.388943 17804 raft_consensus.cc:374] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:29.389031 17804 raft_consensus.cc:729] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14d5204fe7f4f14a2fad1a9046e561c, State: Initialized, Role: FOLLOWER
I20260809 15:13:29.389276 17804 consensus_queue.cc:260] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.389946 17804 ts_tablet_manager.cc:1428] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 15:13:29.399271 17863 tablet_replica.cc:330] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: stopping tablet replica
I20260809 15:13:29.399863 17863 raft_consensus.cc:2229] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:29.400344 17863 raft_consensus.cc:2258] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:29.403712 17863 ts_tablet_manager.cc:1905] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:29.404456 17862 tablet_replica.cc:330] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: stopping tablet replica
I20260809 15:13:29.410844 17862 raft_consensus.cc:2229] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:13:29.413205 17862 raft_consensus.cc:2258] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:29.415143 17862 ts_tablet_manager.cc:1905] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:29.423019 17862 ts_tablet_manager.cc:1918] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:13:29.423247 17862 log.cc:1196] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/f93148cc8e344f3783a6393efc930a5d
I20260809 15:13:29.423887 17862 ts_tablet_manager.cc:1939] T f93148cc8e344f3783a6393efc930a5d P 09e1149656794a05ab3e93dfea17390f: Deleting consensus metadata
I20260809 15:13:29.424019 17863 ts_tablet_manager.cc:1918] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:13:29.424254 17863 log.cc:1196] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/f93148cc8e344f3783a6393efc930a5d
I20260809 15:13:29.424829 17863 ts_tablet_manager.cc:1939] T f93148cc8e344f3783a6393efc930a5d P 9471b7da2c4d42c283f9bcd57e590432: Deleting consensus metadata
I20260809 15:13:29.426793 17236 catalog_manager.cc:4841] TS 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477): tablet f93148cc8e344f3783a6393efc930a5d (table test-table [id=e302d72800a34f039ef81280625000a3]) successfully deleted
I20260809 15:13:29.427162 17234 catalog_manager.cc:4841] TS 09e1149656794a05ab3e93dfea17390f (127.15.11.131:43567): tablet f93148cc8e344f3783a6393efc930a5d (table test-table [id=e302d72800a34f039ef81280625000a3]) successfully deleted
I20260809 15:13:29.547688 17814 raft_consensus.cc:482] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:29.547945 17814 raft_consensus.cc:504] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.548807 17814 leader_election.cc:288] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477), d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469)
I20260809 15:13:29.549517 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7534018f116f43a39223e96f157b7a03" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9471b7da2c4d42c283f9bcd57e590432" is_pre_election: true
I20260809 15:13:29.549773 17419 raft_consensus.cc:2454] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 0.
I20260809 15:13:29.549700 17577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7534018f116f43a39223e96f157b7a03" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" is_pre_election: true
I20260809 15:13:29.549988 17577 raft_consensus.cc:2454] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 0.
I20260809 15:13:29.550400 17651 leader_election.cc:302] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e1149656794a05ab3e93dfea17390f, 9471b7da2c4d42c283f9bcd57e590432; no voters:
I20260809 15:13:29.550791 17814 raft_consensus.cc:2781] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:29.550908 17814 raft_consensus.cc:482] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:29.551043 17814 raft_consensus.cc:3037] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:29.554762 17814 raft_consensus.cc:504] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.555938 17419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7534018f116f43a39223e96f157b7a03" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9471b7da2c4d42c283f9bcd57e590432"
I20260809 15:13:29.556023 17577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7534018f116f43a39223e96f157b7a03" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14d5204fe7f4f14a2fad1a9046e561c"
I20260809 15:13:29.556229 17419 raft_consensus.cc:3037] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:29.556315 17577 raft_consensus.cc:3037] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:29.559429 17419 raft_consensus.cc:2454] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 1.
I20260809 15:13:29.559567 17577 raft_consensus.cc:2454] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 1.
I20260809 15:13:29.559919 17651 leader_election.cc:302] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e1149656794a05ab3e93dfea17390f, 9471b7da2c4d42c283f9bcd57e590432; no voters:
I20260809 15:13:29.560252 17855 raft_consensus.cc:2781] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:29.560434 17855 raft_consensus.cc:686] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 1 LEADER]: Becoming Leader. State: Replica: 09e1149656794a05ab3e93dfea17390f, State: Running, Role: LEADER
I20260809 15:13:29.560559 17814 leader_election.cc:288] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 election: Requested vote from peers 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477), d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469)
I20260809 15:13:29.560779 17855 consensus_queue.cc:237] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:29.570099 17250 catalog_manager.cc:5495] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f reported cstate change: term changed from 0 to 1, leader changed from <none> to 09e1149656794a05ab3e93dfea17390f (127.15.11.131). New cstate: current_term: 1 leader_uuid: "09e1149656794a05ab3e93dfea17390f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } health_report { overall_health: HEALTHY } } }
I20260809 15:13:29.625438 17418 raft_consensus.cc:1261] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Refusing update from remote peer 09e1149656794a05ab3e93dfea17390f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:29.626215 17854 consensus_queue.cc:1035] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:29.628897 17577 raft_consensus.cc:1261] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Refusing update from remote peer 09e1149656794a05ab3e93dfea17390f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:29.629577 17855 consensus_queue.cc:1035] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:13:29.651232 17837 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 15:13:31.232100 17250 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:31.232340 17250 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:31.235965 17250 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f: Sending DeleteTablet for 3 replicas of tablet 7534018f116f43a39223e96f157b7a03
I20260809 15:13:31.236688 17396 tablet_service.cc:1514] Processing DeleteTablet for tablet 7534018f116f43a39223e96f157b7a03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:31 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:42544
I20260809 15:13:31.236737 17549 tablet_service.cc:1514] Processing DeleteTablet for tablet 7534018f116f43a39223e96f157b7a03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:31 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:54702
I20260809 15:13:31.236752 17702 tablet_service.cc:1514] Processing DeleteTablet for tablet 7534018f116f43a39223e96f157b7a03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:31 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:44400
I20260809 15:13:31.247006 17896 tablet_replica.cc:330] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: stopping tablet replica
I20260809 15:13:31.247324 17896 raft_consensus.cc:2229] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:31.247641 17896 raft_consensus.cc:2258] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:31.248898 17894 tablet_replica.cc:330] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: stopping tablet replica
I20260809 15:13:31.248932 17896 ts_tablet_manager.cc:1905] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:31.249114 17894 raft_consensus.cc:2229] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:31.249405 17894 raft_consensus.cc:2258] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:31.249898 17895 tablet_replica.cc:330] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: stopping tablet replica
I20260809 15:13:31.250144 17895 raft_consensus.cc:2229] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:13:31.250514 17894 ts_tablet_manager.cc:1905] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:31.250548 17895 raft_consensus.cc:2258] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:31.251499 17895 ts_tablet_manager.cc:1905] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:31.256577 17896 ts_tablet_manager.cc:1918] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:13:31.256814 17896 log.cc:1196] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/7534018f116f43a39223e96f157b7a03
I20260809 15:13:31.257370 17896 ts_tablet_manager.cc:1939] T 7534018f116f43a39223e96f157b7a03 P d14d5204fe7f4f14a2fad1a9046e561c: Deleting consensus metadata
I20260809 15:13:31.258172 17895 ts_tablet_manager.cc:1918] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:13:31.258352 17895 log.cc:1196] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/7534018f116f43a39223e96f157b7a03
I20260809 15:13:31.258349 17894 ts_tablet_manager.cc:1918] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:13:31.258499 17894 log.cc:1196] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/7534018f116f43a39223e96f157b7a03
I20260809 15:13:31.258667 17237 catalog_manager.cc:4841] TS d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469): tablet 7534018f116f43a39223e96f157b7a03 (table test-table [id=1070f44add594410a821de064ac587c8]) successfully deleted
I20260809 15:13:31.258865 17895 ts_tablet_manager.cc:1939] T 7534018f116f43a39223e96f157b7a03 P 09e1149656794a05ab3e93dfea17390f: Deleting consensus metadata
I20260809 15:13:31.259004 17894 ts_tablet_manager.cc:1939] T 7534018f116f43a39223e96f157b7a03 P 9471b7da2c4d42c283f9bcd57e590432: Deleting consensus metadata
I20260809 15:13:31.259972 17234 catalog_manager.cc:4841] TS 09e1149656794a05ab3e93dfea17390f (127.15.11.131:43567): tablet 7534018f116f43a39223e96f157b7a03 (table test-table [id=1070f44add594410a821de064ac587c8]) successfully deleted
I20260809 15:13:31.260020 17236 catalog_manager.cc:4841] TS 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477): tablet 7534018f116f43a39223e96f157b7a03 (table test-table [id=1070f44add594410a821de064ac587c8]) successfully deleted
Aug 09 15:13:31 dist-test-slave-8w9v krb5kdc[17203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:13:31 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:31.294849 17905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.280795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35048 (local address 127.15.11.190:43717)
0809 15:13:31.281397 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:31.281403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:31.281487 (+ 84us) server_negotiation.cc:407] Connection header received
0809 15:13:31.281717 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:31.281721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:31.281786 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:31.281898 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:31.283226 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.283935 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.285242 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.285674 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.285812 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.289608 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:31.289612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:31.289616 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:31.289648 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:31.292426 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:31.293141 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:31.293144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:31.293146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:31.293263 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:31.293558 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:31.293561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:31.293563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:31.293837 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:31.294048 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:31.294287 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 15:13:31.294460 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":266,"threads_started":1}
I20260809 15:13:31.298106 17250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:13:31.298640 17250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:31.305048 17396 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7bba4c41e648c2b5810057ac56ce6f (DEFAULT_TABLE table=test-table [id=2251cb97778e481fbaf37deb0b9ccfbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:31.305142 17549 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7bba4c41e648c2b5810057ac56ce6f (DEFAULT_TABLE table=test-table [id=2251cb97778e481fbaf37deb0b9ccfbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:31.305346 17702 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7bba4c41e648c2b5810057ac56ce6f (DEFAULT_TABLE table=test-table [id=2251cb97778e481fbaf37deb0b9ccfbf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:31.305540 17396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7bba4c41e648c2b5810057ac56ce6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:31.305657 17549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7bba4c41e648c2b5810057ac56ce6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:31.305828 17702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7bba4c41e648c2b5810057ac56ce6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:31.312021 17906 tablet_bootstrap.cc:492] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: Bootstrap starting.
I20260809 15:13:31.313550 17906 tablet_bootstrap.cc:654] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:31.316315 17906 tablet_bootstrap.cc:492] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: No bootstrap required, opened a new log
I20260809 15:13:31.316465 17906 ts_tablet_manager.cc:1397] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260809 15:13:31.316624 17907 tablet_bootstrap.cc:492] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: Bootstrap starting.
I20260809 15:13:31.317202 17906 raft_consensus.cc:348] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.317374 17906 raft_consensus.cc:374] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:31.317463 17906 raft_consensus.cc:729] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9471b7da2c4d42c283f9bcd57e590432, State: Initialized, Role: FOLLOWER
I20260809 15:13:31.317478 17908 tablet_bootstrap.cc:492] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: Bootstrap starting.
I20260809 15:13:31.317689 17906 consensus_queue.cc:260] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.318063 17907 tablet_bootstrap.cc:654] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:31.319157 17908 tablet_bootstrap.cc:654] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:31.319679 17906 ts_tablet_manager.cc:1428] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 15:13:31.321126 17907 tablet_bootstrap.cc:492] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: No bootstrap required, opened a new log
I20260809 15:13:31.321295 17907 ts_tablet_manager.cc:1397] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:13:31.321549 17908 tablet_bootstrap.cc:492] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: No bootstrap required, opened a new log
I20260809 15:13:31.321695 17908 ts_tablet_manager.cc:1397] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 15:13:31.322077 17907 raft_consensus.cc:348] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.322307 17907 raft_consensus.cc:374] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:31.322431 17907 raft_consensus.cc:729] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09e1149656794a05ab3e93dfea17390f, State: Initialized, Role: FOLLOWER
I20260809 15:13:31.322533 17908 raft_consensus.cc:348] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.322685 17907 consensus_queue.cc:260] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.322866 17908 raft_consensus.cc:374] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:31.323256 17907 ts_tablet_manager.cc:1428] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:13:31.322959 17908 raft_consensus.cc:729] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14d5204fe7f4f14a2fad1a9046e561c, State: Initialized, Role: FOLLOWER
I20260809 15:13:31.323716 17908 consensus_queue.cc:260] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.324354 17908 ts_tablet_manager.cc:1428] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:31.333956 17885 raft_consensus.cc:482] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:31.334133 17885 raft_consensus.cc:504] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.334769 17885 leader_election.cc:288] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477), d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469)
I20260809 15:13:31.335197 17418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7bba4c41e648c2b5810057ac56ce6f" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9471b7da2c4d42c283f9bcd57e590432" is_pre_election: true
I20260809 15:13:31.335304 17577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7bba4c41e648c2b5810057ac56ce6f" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" is_pre_election: true
I20260809 15:13:31.335481 17418 raft_consensus.cc:2454] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 0.
I20260809 15:13:31.335582 17577 raft_consensus.cc:2454] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 0.
I20260809 15:13:31.335955 17651 leader_election.cc:302] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e1149656794a05ab3e93dfea17390f, 9471b7da2c4d42c283f9bcd57e590432; no voters:
I20260809 15:13:31.336241 17885 raft_consensus.cc:2781] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:31.336380 17885 raft_consensus.cc:482] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:31.336498 17885 raft_consensus.cc:3037] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:31.338569 17885 raft_consensus.cc:504] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.339058 17885 leader_election.cc:288] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 election: Requested vote from peers 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477), d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469)
I20260809 15:13:31.339499 17418 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7bba4c41e648c2b5810057ac56ce6f" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9471b7da2c4d42c283f9bcd57e590432"
I20260809 15:13:31.339547 17577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7bba4c41e648c2b5810057ac56ce6f" candidate_uuid: "09e1149656794a05ab3e93dfea17390f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14d5204fe7f4f14a2fad1a9046e561c"
I20260809 15:13:31.339690 17418 raft_consensus.cc:3037] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:31.339745 17577 raft_consensus.cc:3037] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:31.341802 17577 raft_consensus.cc:2454] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 1.
I20260809 15:13:31.342057 17418 raft_consensus.cc:2454] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09e1149656794a05ab3e93dfea17390f in term 1.
I20260809 15:13:31.342252 17652 leader_election.cc:302] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e1149656794a05ab3e93dfea17390f, d14d5204fe7f4f14a2fad1a9046e561c; no voters:
I20260809 15:13:31.342555 17885 raft_consensus.cc:2781] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:31.342700 17885 raft_consensus.cc:686] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 1 LEADER]: Becoming Leader. State: Replica: 09e1149656794a05ab3e93dfea17390f, State: Running, Role: LEADER
I20260809 15:13:31.342983 17885 consensus_queue.cc:237] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.345769 17250 catalog_manager.cc:5495] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f reported cstate change: term changed from 0 to 1, leader changed from <none> to 09e1149656794a05ab3e93dfea17390f (127.15.11.131). New cstate: current_term: 1 leader_uuid: "09e1149656794a05ab3e93dfea17390f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } health_report { overall_health: HEALTHY } } }
I20260809 15:13:31.373443 17916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.368639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44412 (local address 127.15.11.131:43567)
0809 15:13:31.369226 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:31.369229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:31.369246 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:13:31.369294 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:31.369297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:31.369349 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:31.369429 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:31.369734 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.370446 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.371710 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.372100 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.372222 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.372780 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:31.372877 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:31.372970 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:31.373125 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 15:13:31.373185 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":374,"threads_started":1}
I20260809 15:13:31.382519 17577 raft_consensus.cc:1261] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Refusing update from remote peer 09e1149656794a05ab3e93dfea17390f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:31.382530 17418 raft_consensus.cc:1261] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Refusing update from remote peer 09e1149656794a05ab3e93dfea17390f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:31.383049 17855 consensus_queue.cc:1035] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:31.383329 17885 consensus_queue.cc:1035] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:31.391301 17874 mvcc.cc:205] Tried to move back new op lower bound from 7316637333018148864 to 7316637332863414272. Current Snapshot: MvccSnapshot[applied={T|T < 7316637333018148864}]
I20260809 15:13:31.430423 17251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:31.430675 17251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:31.433547 17251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f: Sending DeleteTablet for 3 replicas of tablet 2f7bba4c41e648c2b5810057ac56ce6f
I20260809 15:13:31.434409 17549 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f7bba4c41e648c2b5810057ac56ce6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:31 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:54702
I20260809 15:13:31.434402 17396 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f7bba4c41e648c2b5810057ac56ce6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:31 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:42544
I20260809 15:13:31.434473 17702 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f7bba4c41e648c2b5810057ac56ce6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:31 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:44400
I20260809 15:13:31.434695 17894 tablet_replica.cc:330] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: stopping tablet replica
I20260809 15:13:31.434708 17895 tablet_replica.cc:330] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: stopping tablet replica
I20260809 15:13:31.434907 17895 raft_consensus.cc:2229] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:13:31.434906 17894 raft_consensus.cc:2229] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:31.435094 17894 raft_consensus.cc:2258] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:31.435204 17896 tablet_replica.cc:330] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: stopping tablet replica
I20260809 15:13:31.435237 17895 raft_consensus.cc:2258] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:31.435448 17896 raft_consensus.cc:2229] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:31.435639 17896 raft_consensus.cc:2258] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:31.436326 17894 ts_tablet_manager.cc:1905] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:31.436398 17895 ts_tablet_manager.cc:1905] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:31.436693 17251 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:13:31.437258 17251 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:31.444000 17702 tablet_service.cc:1467] Processing CreateTablet for tablet 9d74c03e7b4f4a7eae7f01619e583dc5 (DEFAULT_TABLE table=test-table [id=c66ad0bd370947c28704c26917a5b53b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:31.444310 17549 tablet_service.cc:1467] Processing CreateTablet for tablet 9d74c03e7b4f4a7eae7f01619e583dc5 (DEFAULT_TABLE table=test-table [id=c66ad0bd370947c28704c26917a5b53b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:31.444471 17702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d74c03e7b4f4a7eae7f01619e583dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:31.444697 17549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d74c03e7b4f4a7eae7f01619e583dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:31.448295 17895 ts_tablet_manager.cc:1918] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:13:31.448395 17396 tablet_service.cc:1467] Processing CreateTablet for tablet 9d74c03e7b4f4a7eae7f01619e583dc5 (DEFAULT_TABLE table=test-table [id=c66ad0bd370947c28704c26917a5b53b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:31.448503 17895 log.cc:1196] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/2f7bba4c41e648c2b5810057ac56ce6f
I20260809 15:13:31.448853 17396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d74c03e7b4f4a7eae7f01619e583dc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:31.449220 17895 ts_tablet_manager.cc:1939] T 2f7bba4c41e648c2b5810057ac56ce6f P 09e1149656794a05ab3e93dfea17390f: Deleting consensus metadata
I20260809 15:13:31.450003 17908 tablet_bootstrap.cc:492] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: Bootstrap starting.
I20260809 15:13:31.450953 17896 ts_tablet_manager.cc:1905] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:31.451998 17908 tablet_bootstrap.cc:654] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:31.453332 17234 catalog_manager.cc:4841] TS 09e1149656794a05ab3e93dfea17390f (127.15.11.131:43567): tablet 2f7bba4c41e648c2b5810057ac56ce6f (table test-table [id=2251cb97778e481fbaf37deb0b9ccfbf]) successfully deleted
I20260809 15:13:31.453711 17894 ts_tablet_manager.cc:1918] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:13:31.453927 17894 log.cc:1196] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/2f7bba4c41e648c2b5810057ac56ce6f
I20260809 15:13:31.454622 17894 ts_tablet_manager.cc:1939] T 2f7bba4c41e648c2b5810057ac56ce6f P 9471b7da2c4d42c283f9bcd57e590432: Deleting consensus metadata
I20260809 15:13:31.455633 17236 catalog_manager.cc:4841] TS 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477): tablet 2f7bba4c41e648c2b5810057ac56ce6f (table test-table [id=2251cb97778e481fbaf37deb0b9ccfbf]) successfully deleted
I20260809 15:13:31.456041 17906 tablet_bootstrap.cc:492] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: Bootstrap starting.
I20260809 15:13:31.457623 17907 tablet_bootstrap.cc:492] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: Bootstrap starting.
I20260809 15:13:31.457876 17906 tablet_bootstrap.cc:654] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:31.459405 17896 ts_tablet_manager.cc:1918] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:13:31.459574 17896 log.cc:1196] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/2f7bba4c41e648c2b5810057ac56ce6f
I20260809 15:13:31.459882 17907 tablet_bootstrap.cc:654] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:31.460068 17896 ts_tablet_manager.cc:1939] T 2f7bba4c41e648c2b5810057ac56ce6f P d14d5204fe7f4f14a2fad1a9046e561c: Deleting consensus metadata
I20260809 15:13:31.461716 17237 catalog_manager.cc:4841] TS d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469): tablet 2f7bba4c41e648c2b5810057ac56ce6f (table test-table [id=2251cb97778e481fbaf37deb0b9ccfbf]) successfully deleted
I20260809 15:13:31.462430 17908 tablet_bootstrap.cc:492] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: No bootstrap required, opened a new log
I20260809 15:13:31.462610 17908 ts_tablet_manager.cc:1397] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20260809 15:13:31.463397 17908 raft_consensus.cc:348] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.463606 17908 raft_consensus.cc:374] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:31.463778 17908 raft_consensus.cc:729] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14d5204fe7f4f14a2fad1a9046e561c, State: Initialized, Role: FOLLOWER
I20260809 15:13:31.464011 17908 consensus_queue.cc:260] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.464524 17908 ts_tablet_manager.cc:1428] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:13:31.466959 17907 tablet_bootstrap.cc:492] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: No bootstrap required, opened a new log
I20260809 15:13:31.467162 17907 ts_tablet_manager.cc:1397] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.007s
I20260809 15:13:31.468039 17907 raft_consensus.cc:348] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.468353 17907 raft_consensus.cc:374] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:31.468427 17907 raft_consensus.cc:729] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09e1149656794a05ab3e93dfea17390f, State: Initialized, Role: FOLLOWER
I20260809 15:13:31.468719 17907 consensus_queue.cc:260] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.469394 17907 ts_tablet_manager.cc:1428] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 15:13:31.471009 17906 tablet_bootstrap.cc:492] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: No bootstrap required, opened a new log
I20260809 15:13:31.471220 17906 ts_tablet_manager.cc:1397] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.004s
I20260809 15:13:31.471804 17906 raft_consensus.cc:348] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.472031 17906 raft_consensus.cc:374] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:31.472153 17906 raft_consensus.cc:729] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9471b7da2c4d42c283f9bcd57e590432, State: Initialized, Role: FOLLOWER
I20260809 15:13:31.472414 17906 consensus_queue.cc:260] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.473094 17906 ts_tablet_manager.cc:1428] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260809 15:13:31.516577 17910 raft_consensus.cc:482] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:31.516791 17910 raft_consensus.cc:504] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.518297 17910 leader_election.cc:288] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469), 09e1149656794a05ab3e93dfea17390f (127.15.11.131:43567)
I20260809 15:13:31.526356 17916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.518708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:46395 (local address 127.15.11.131:43567)
0809 15:13:31.518906 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:31.518914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:31.519966 (+ 1052us) server_negotiation.cc:407] Connection header received
0809 15:13:31.520105 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:31.520110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:31.520222 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:31.520324 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:31.521298 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.522401 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.524621 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.525393 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.525555 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.525632 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:31.525982 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 15:13:31.526126 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:13:31.526414 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.518682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:37469 (local address 127.15.11.129:60721)
0809 15:13:31.519656 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:31.519678 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:31.519840 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:31.520308 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:31.520313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:31.520348 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:31.520823 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:31.520840 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.522509 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.522513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:31.524177 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:31.524194 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.525501 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.525508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:31.525720 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.525742 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:13:31.525835 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 15:13:31.525955 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":2,"thread_start_us":641,"threads_started":1}
I20260809 15:13:31.526422 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.518757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:60721 (local address 127.15.11.130:37469)
0809 15:13:31.519398 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:31.519405 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:31.519784 (+ 379us) server_negotiation.cc:407] Connection header received
0809 15:13:31.520046 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:31.520052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:31.520159 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:31.520279 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:31.521066 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.522350 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.524374 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.525294 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.525434 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.525546 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:31.525903 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 15:13:31.526057 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":337,"threads_started":1}
I20260809 15:13:31.526532 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.518709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:43567 (local address 127.15.11.129:46395)
0809 15:13:31.519849 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:31.519868 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:31.519986 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:31.520518 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:31.520525 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:31.520554 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:31.521055 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:31.521069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.522642 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.522646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:31.524462 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:31.524479 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.525628 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.525634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:31.525785 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.525809 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:13:31.525934 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 15:13:31.526057 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1011,"mutex_wait_us":20,"thread_start_us":881,"threads_started":1}
I20260809 15:13:31.527197 17730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d74c03e7b4f4a7eae7f01619e583dc5" candidate_uuid: "9471b7da2c4d42c283f9bcd57e590432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09e1149656794a05ab3e93dfea17390f" is_pre_election: true
I20260809 15:13:31.527257 17577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d74c03e7b4f4a7eae7f01619e583dc5" candidate_uuid: "9471b7da2c4d42c283f9bcd57e590432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" is_pre_election: true
I20260809 15:13:31.527451 17730 raft_consensus.cc:2454] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9471b7da2c4d42c283f9bcd57e590432 in term 0.
I20260809 15:13:31.527515 17577 raft_consensus.cc:2454] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9471b7da2c4d42c283f9bcd57e590432 in term 0.
I20260809 15:13:31.527995 17343 leader_election.cc:302] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e1149656794a05ab3e93dfea17390f, 9471b7da2c4d42c283f9bcd57e590432; no voters:
I20260809 15:13:31.528458 17910 raft_consensus.cc:2781] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:31.528594 17910 raft_consensus.cc:482] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:31.528713 17910 raft_consensus.cc:3037] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:31.530968 17910 raft_consensus.cc:504] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.531648 17910 leader_election.cc:288] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [CANDIDATE]: Term 1 election: Requested vote from peers d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469), 09e1149656794a05ab3e93dfea17390f (127.15.11.131:43567)
I20260809 15:13:31.532074 17577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d74c03e7b4f4a7eae7f01619e583dc5" candidate_uuid: "9471b7da2c4d42c283f9bcd57e590432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14d5204fe7f4f14a2fad1a9046e561c"
I20260809 15:13:31.532109 17730 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d74c03e7b4f4a7eae7f01619e583dc5" candidate_uuid: "9471b7da2c4d42c283f9bcd57e590432" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09e1149656794a05ab3e93dfea17390f"
I20260809 15:13:31.532347 17577 raft_consensus.cc:3037] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:31.532387 17730 raft_consensus.cc:3037] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:31.534416 17730 raft_consensus.cc:2454] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9471b7da2c4d42c283f9bcd57e590432 in term 1.
I20260809 15:13:31.534489 17577 raft_consensus.cc:2454] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9471b7da2c4d42c283f9bcd57e590432 in term 1.
I20260809 15:13:31.534875 17343 leader_election.cc:302] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09e1149656794a05ab3e93dfea17390f, 9471b7da2c4d42c283f9bcd57e590432; no voters:
I20260809 15:13:31.535220 17910 raft_consensus.cc:2781] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:31.535681 17910 raft_consensus.cc:686] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 LEADER]: Becoming Leader. State: Replica: 9471b7da2c4d42c283f9bcd57e590432, State: Running, Role: LEADER
I20260809 15:13:31.536092 17910 consensus_queue.cc:237] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } }
I20260809 15:13:31.539852 17251 catalog_manager.cc:5495] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129). New cstate: current_term: 1 leader_uuid: "9471b7da2c4d42c283f9bcd57e590432" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9471b7da2c4d42c283f9bcd57e590432" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 } health_report { overall_health: UNKNOWN } } }
I20260809 15:13:31.573393 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.567990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42558 (local address 127.15.11.129:39477)
0809 15:13:31.568686 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:31.568690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:31.568709 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:13:31.568787 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:31.568791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:31.568857 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:31.569003 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:31.569303 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.570060 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.571260 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.571648 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.571736 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.572180 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:31.572817 (+ 637us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:31.572996 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:31.573041 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 15:13:31.573105 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":444,"threads_started":1}
I20260809 15:13:31.587896 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.583171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:57939 (local address 127.15.11.130:37469)
0809 15:13:31.583369 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:31.583377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:31.583923 (+ 546us) server_negotiation.cc:407] Connection header received
0809 15:13:31.584057 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:31.584061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:31.584171 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:31.584244 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:31.584705 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.585366 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.586540 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.586929 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.587012 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.587379 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:31.587471 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:31.587532 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:31.587652 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 15:13:31.587707 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:13:31.588047 17932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.583215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:37469 (local address 127.15.11.129:57939)
0809 15:13:31.583814 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:31.583834 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:31.583955 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:31.584272 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:31.584276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:31.584285 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:13:31.584567 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:31.584573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.585481 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.585485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:31.586432 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:31.586443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.587068 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.587072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:31.587191 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.587239 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:13:31.587552 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:13:31.587560 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:13:31.587616 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 15:13:31.587708 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":304,"threads_started":1}
I20260809 15:13:31.595887 17730 raft_consensus.cc:1261] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Refusing update from remote peer 9471b7da2c4d42c283f9bcd57e590432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:31.595912 17577 raft_consensus.cc:1261] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Refusing update from remote peer 9471b7da2c4d42c283f9bcd57e590432: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:31.596647 17927 consensus_queue.cc:1035] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14d5204fe7f4f14a2fad1a9046e561c" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 37469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:31.596984 17910 consensus_queue.cc:1035] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09e1149656794a05ab3e93dfea17390f" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:31.621469 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.615903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:46299 (local address 127.15.11.129:39477)
0809 15:13:31.616147 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:31.616154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:31.616543 (+ 389us) server_negotiation.cc:407] Connection header received
0809 15:13:31.616732 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:31.616737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:31.616808 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:31.616922 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:31.617324 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.618013 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.619772 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.620267 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.620401 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.620781 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:31.620889 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:31.620986 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:31.621122 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 15:13:31.621194 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 15:13:31.621579 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:31.615732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:39477 (local address 127.15.11.130:46299)
0809 15:13:31.616439 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:31.616458 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:31.616593 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:31.616949 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:31.616954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:31.616980 (+ 26us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:13:31.617191 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:31.617199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.618155 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.618160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:31.619631 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:31.619646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:31.620444 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:31.620452 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:31.620591 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:31.620625 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:13:31.620988 (+ 363us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:13:31.621001 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 15:13:31.621100 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 15:13:31.621224 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":357,"threads_started":1}
W20260809 15:13:31.622475 17874 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 15:13:33.220837 17251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:33.221056 17251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:33.224963 17251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e1c068ee01b04857afa59fa90b2ebd0f: Sending DeleteTablet for 3 replicas of tablet 9d74c03e7b4f4a7eae7f01619e583dc5
I20260809 15:13:33.225726 17550 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d74c03e7b4f4a7eae7f01619e583dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:33 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:54702
I20260809 15:13:33.225817 17702 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d74c03e7b4f4a7eae7f01619e583dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:33 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:44400
I20260809 15:13:33.226208 17396 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d74c03e7b4f4a7eae7f01619e583dc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:33 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:42544
I20260809 15:13:33.226356 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 17325
I20260809 15:13:33.226483 17966 tablet_replica.cc:330] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: stopping tablet replica
I20260809 15:13:33.226734 17966 raft_consensus.cc:2229] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:33.227077 17966 raft_consensus.cc:2258] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:33.228150 17966 ts_tablet_manager.cc:1905] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:33.237275 17966 ts_tablet_manager.cc:1918] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:13:33.239570 17966 log.cc:1196] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/9d74c03e7b4f4a7eae7f01619e583dc5
I20260809 15:13:33.239727 17969 tablet_replica.cc:330] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: stopping tablet replica
I20260809 15:13:33.239763 17968 tablet_replica.cc:330] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: stopping tablet replica
I20260809 15:13:33.239950 17968 raft_consensus.cc:2229] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:33.240036 17969 raft_consensus.cc:2229] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:13:33.240105 17966 ts_tablet_manager.cc:1939] T 9d74c03e7b4f4a7eae7f01619e583dc5 P d14d5204fe7f4f14a2fad1a9046e561c: Deleting consensus metadata
I20260809 15:13:33.240273 17968 raft_consensus.cc:2258] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:33.241444 17237 catalog_manager.cc:4841] TS d14d5204fe7f4f14a2fad1a9046e561c (127.15.11.130:37469): tablet 9d74c03e7b4f4a7eae7f01619e583dc5 (table test-table [id=c66ad0bd370947c28704c26917a5b53b]) successfully deleted
I20260809 15:13:33.241586 17969 raft_consensus.cc:2258] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:33.242014 17968 ts_tablet_manager.cc:1905] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:33.242652 17969 ts_tablet_manager.cc:1905] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 15:13:33 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:33.248811 17968 ts_tablet_manager.cc:1918] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:13:33.248943 17968 log.cc:1196] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/9d74c03e7b4f4a7eae7f01619e583dc5
I20260809 15:13:33.249411 17968 ts_tablet_manager.cc:1939] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 09e1149656794a05ab3e93dfea17390f: Deleting consensus metadata
I20260809 15:13:33.249404 17969 ts_tablet_manager.cc:1918] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:13:33.249558 17969 log.cc:1196] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/9d74c03e7b4f4a7eae7f01619e583dc5
I20260809 15:13:33.250063 17969 ts_tablet_manager.cc:1939] T 9d74c03e7b4f4a7eae7f01619e583dc5 P 9471b7da2c4d42c283f9bcd57e590432: Deleting consensus metadata
I20260809 15:13:33.250735 17234 catalog_manager.cc:4841] TS 09e1149656794a05ab3e93dfea17390f (127.15.11.131:43567): tablet 9d74c03e7b4f4a7eae7f01619e583dc5 (table test-table [id=c66ad0bd370947c28704c26917a5b53b]) successfully deleted
I20260809 15:13:33.251178 17236 catalog_manager.cc:4841] TS 9471b7da2c4d42c283f9bcd57e590432 (127.15.11.129:39477): tablet 9d74c03e7b4f4a7eae7f01619e583dc5 (table test-table [id=c66ad0bd370947c28704c26917a5b53b]) successfully deleted
I20260809 15:13:33.255267 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:33.229268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42568 (local address 127.15.11.129:39477)
0809 15:13:33.239600 (+ 10332us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:33.239606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:33.239631 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:13:33.239699 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:33.239706 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:33.239816 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:33.240618 (+ 802us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:33.241754 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:33.242709 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:33.244178 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:33.244732 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:33.244865 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:33.249130 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:33.249152 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:33.249169 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:33.249199 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:33.252697 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:33.253339 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:33.253347 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:33.253349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:33.253501 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:33.253901 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:33.253905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:33.253907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:33.254269 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:33.254487 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:33.254731 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 15:13:33.254900 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10179,"thread_start_us":9983,"threads_started":1}
W20260809 15:13:33.255930 17469 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42568
W20260809 15:13:33.256589 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 17325: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:13:33.256773 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 17325
I20260809 15:13:33.285475 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 17478
Aug 09 15:13:33 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:13:33.300388 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:33.286265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.15.11.130:37469)
0809 15:13:33.286724 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:33.286728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:33.286743 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:13:33.286800 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:33.286804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:33.286873 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:33.286975 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:33.288223 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:33.288950 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:33.290165 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:33.290651 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:33.290745 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:33.295039 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:33.295050 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:33.295076 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:33.295108 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:33.298007 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:33.298600 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:33.298606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:33.298608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:33.298728 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:33.299034 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:33.299037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:33.299039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:33.299373 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:33.299594 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:33.299829 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 15:13:33.299999 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":237,"threads_started":1}
W20260809 15:13:33.301075 17622 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54720
W20260809 15:13:33.301656 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 17478: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:13:33.301813 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 17478
I20260809 15:13:33.328665 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 17631
Aug 09 15:13:33 dist-test-slave-8w9v krb5kdc[17203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:13:33.343433 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:33.329457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44422 (local address 127.15.11.131:43567)
0809 15:13:33.329934 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:33.329938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:33.329954 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:13:33.330008 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:33.330012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:33.330075 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:33.330167 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:33.331415 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:33.332251 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:33.333412 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:33.333810 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:33.333906 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:33.338063 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:33.338078 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:33.338095 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:33.338129 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:33.341128 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:33.341734 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:33.341742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:33.341744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:33.341883 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:33.342195 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:33.342199 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:33.342200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:33.342547 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:33.342793 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:33.342978 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:13:33.343145 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":241,"threads_started":1}
W20260809 15:13:33.344184 17777 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44422
W20260809 15:13:33.344767 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 17631: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:13:33.344911 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 17631
I20260809 15:13:33.372620 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 17216
I20260809 15:13:33.373839 17292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:33.492901 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 17216
2026-08-09T15:13:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8450 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:33 dist-test-slave-8w9v krb5kdc[17978](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:33 dist-test-slave-8w9v krb5kdc[17978](info): set up 2 sockets
Aug 09 15:13:33 dist-test-slave-8w9v krb5kdc[17978](info): commencing operation
krb5kdc: starting...
W20260809 15:13:34.625309 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:34 dist-test-slave-8w9v krb5kdc[17978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:34.826056 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:44935
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:44935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:35.149101 17991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:35.149430 17991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:35.149613 17991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:35.156517 17991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:35.156648 17991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:35.156754 17991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:35.156854 17991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:35.156920 17991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:35.166121 17991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:44935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:44935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:35.167393 17991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:35.168756 17991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:35.180828 17999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:35.180876 17996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:35.180917 17997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:35.182174 17991 server_base.cc:1034] running on GCE node
I20260809 15:13:35.182937 17991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:35.184396 17991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:35.185595 17991 hybrid_clock.cc:648] HybridClock initialized: now 1786288415185552 us; error 30 us; skew 500 ppm
I20260809 15:13:35.185976 17991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:35 dist-test-slave-8w9v krb5kdc[17978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288415, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:35.188999 17991 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:35.190425 17991 webserver.cc:460] Webserver started at http://127.15.11.190:33365/ using document root <none> and password file <none>
I20260809 15:13:35.191087 17991 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:35.191224 17991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:35.191550 17991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:35.194554 17991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "3053c89c265d44a6987b8964d881ed08"
format_stamp: "Formatted at 2026-08-09 15:13:35 on dist-test-slave-8w9v"
server_key: "fc94bfd47d538abd26fe832762dd3be9"
server_key_iv: "5f12a2f1ef085be5c77ae18dbe10340b"
server_key_version: "encryptionkey@0"
I20260809 15:13:35.195305 17991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "3053c89c265d44a6987b8964d881ed08"
format_stamp: "Formatted at 2026-08-09 15:13:35 on dist-test-slave-8w9v"
server_key: "fc94bfd47d538abd26fe832762dd3be9"
server_key_iv: "5f12a2f1ef085be5c77ae18dbe10340b"
server_key_version: "encryptionkey@0"
I20260809 15:13:35.200523 17991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 15:13:35.203846 18006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:35.205164 17991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:35.205353 17991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "3053c89c265d44a6987b8964d881ed08"
format_stamp: "Formatted at 2026-08-09 15:13:35 on dist-test-slave-8w9v"
server_key: "fc94bfd47d538abd26fe832762dd3be9"
server_key_iv: "5f12a2f1ef085be5c77ae18dbe10340b"
server_key_version: "encryptionkey@0"
I20260809 15:13:35.205556 17991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:35.225069 17991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:35.228937 17991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:35.229228 17991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:35.247539 17991 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:44935
I20260809 15:13:35.247571 18066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:44935 every 8 connection(s)
I20260809 15:13:35.248950 17991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:35.252715 18067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:35.254717 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 17991
I20260809 15:13:35.255034 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:35.255688 15406 external_mini_cluster.cc:1428] Setting key d6be95fe5779a0970cd4a90d48f711c3
I20260809 15:13:35.264673 18067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08: Bootstrap starting.
I20260809 15:13:35.268505 18067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:35.269795 18067 log.cc:824] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:13:35 dist-test-slave-8w9v krb5kdc[17978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:35.272668 18067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08: No bootstrap required, opened a new log
I20260809 15:13:35.278371 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:35.257765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44192 (local address 127.15.11.190:44935)
0809 15:13:35.258567 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:35.258577 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:35.258617 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:13:35.260238 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:35.260258 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:35.260613 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:35.260932 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:35.262655 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:35.264019 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:35.265789 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:35.266440 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:35.266560 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:35.271374 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:35.271382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:35.271397 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:35.271428 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:35.274428 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:35.275114 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:35.275121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:35.275126 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:35.275293 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:35.275663 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:35.275668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:35.275680 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:35.276233 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:35.276592 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:35.277034 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 15:13:35.277422 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":364,"threads_started":1}
I20260809 15:13:35.280879 18067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3053c89c265d44a6987b8964d881ed08" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44935 } }
I20260809 15:13:35.281333 18067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:35.281440 18067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3053c89c265d44a6987b8964d881ed08, State: Initialized, Role: FOLLOWER
I20260809 15:13:35.281912 18067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3053c89c265d44a6987b8964d881ed08" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44935 } }
I20260809 15:13:35.282097 18067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:35.282186 18067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:35.282303 18067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:35.284343 18067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3053c89c265d44a6987b8964d881ed08" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44935 } }
I20260809 15:13:35.284806 18067 leader_election.cc:302] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3053c89c265d44a6987b8964d881ed08; no voters:
I20260809 15:13:35.285315 18067 leader_election.cc:288] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:35.285471 18072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:35.286684 18072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [term 1 LEADER]: Becoming Leader. State: Replica: 3053c89c265d44a6987b8964d881ed08, State: Running, Role: LEADER
I20260809 15:13:35.287217 18072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3053c89c265d44a6987b8964d881ed08" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44935 } }
I20260809 15:13:35.287596 18067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:35.290481 18074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3053c89c265d44a6987b8964d881ed08. Latest consensus state: current_term: 1 leader_uuid: "3053c89c265d44a6987b8964d881ed08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3053c89c265d44a6987b8964d881ed08" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44935 } } }
I20260809 15:13:35.290464 18073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3053c89c265d44a6987b8964d881ed08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3053c89c265d44a6987b8964d881ed08" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44935 } } }
I20260809 15:13:35.291285 18074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:35.291373 18073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:35.296619 18082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:35.301054 18082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:35.309175 18082 catalog_manager.cc:1347] Generated new cluster ID: 71ee6eb015304e93b1d6fa871b6c8121
I20260809 15:13:35.309265 18082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:35.375609 18082 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:35.377117 18082 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:35.385404 18082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3053c89c265d44a6987b8964d881ed08: Generated new TSK 0
I20260809 15:13:35.386325 18082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:13:35.456110 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:44935
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:35.788897 18096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:35.789216 18096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:35.789397 18096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:35.796727 18096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:35.796842 18096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:35.797011 18096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:35.806272 18096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:44935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:35.807435 18096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:35.808902 18096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:35.821635 18104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:35.821703 18101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:35.821671 18102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:35.822692 18096 server_base.cc:1034] running on GCE node
I20260809 15:13:35.823246 18096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:35.824105 18096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:35.825354 18096 hybrid_clock.cc:648] HybridClock initialized: now 1786288415825304 us; error 45 us; skew 500 ppm
I20260809 15:13:35.825644 18096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:35 dist-test-slave-8w9v krb5kdc[17978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288415, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:35.828698 18096 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:13:35.830072 18096 webserver.cc:460] Webserver started at http://127.15.11.129:36639/ using document root <none> and password file <none>
I20260809 15:13:35.830775 18096 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:35.830909 18096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:35.831192 18096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:35.834113 18096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "e82d77c1339848e198e5678403110d7e"
format_stamp: "Formatted at 2026-08-09 15:13:35 on dist-test-slave-8w9v"
server_key: "52d461736540bd1921b3e8d630d60e74"
server_key_iv: "5fae210264d57ee78ae9d34c9930ca23"
server_key_version: "encryptionkey@0"
I20260809 15:13:35.834926 18096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "e82d77c1339848e198e5678403110d7e"
format_stamp: "Formatted at 2026-08-09 15:13:35 on dist-test-slave-8w9v"
server_key: "52d461736540bd1921b3e8d630d60e74"
server_key_iv: "5fae210264d57ee78ae9d34c9930ca23"
server_key_version: "encryptionkey@0"
I20260809 15:13:35.839924 18096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 15:13:35.843133 18112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:35.844408 18096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:35.844594 18096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "e82d77c1339848e198e5678403110d7e"
format_stamp: "Formatted at 2026-08-09 15:13:35 on dist-test-slave-8w9v"
server_key: "52d461736540bd1921b3e8d630d60e74"
server_key_iv: "5fae210264d57ee78ae9d34c9930ca23"
server_key_version: "encryptionkey@0"
I20260809 15:13:35.844792 18096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:35.868697 18096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:35.872438 18096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:35.872722 18096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:35.873721 18096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:35.875029 18096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:35.875128 18096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:35.875268 18096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:35.875336 18096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:35.908176 18096 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:41895
I20260809 15:13:35.908228 18235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:41895 every 8 connection(s)
I20260809 15:13:35.909472 18096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:13:35.917657 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 18096
I20260809 15:13:35.917889 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:13:35.918718 15406 external_mini_cluster.cc:1428] Setting key 78fe4b594f6a97330b99c2fc1afc245e
Aug 09 15:13:35 dist-test-slave-8w9v krb5kdc[17978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288415, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:35.930687 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:35.911940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:53161 (local address 127.15.11.190:44935)
0809 15:13:35.912439 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:35.912445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:35.913363 (+ 918us) server_negotiation.cc:407] Connection header received
0809 15:13:35.915089 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:35.915093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:35.915160 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:35.915269 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:35.916647 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:35.917316 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:35.919675 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:35.920449 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:35.920612 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:35.924481 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:35.924502 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:35.924506 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:35.924537 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:35.927064 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:35.927783 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:35.927787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:35.927790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:35.927927 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:35.928511 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:35.928516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:35.928519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:35.928877 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:35.929001 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:35.929944 (+ 943us) server_negotiation.cc:299] Negotiation successful
0809 15:13:35.930197 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
I20260809 15:13:35.931201 18239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:35.912517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:44935 (local address 127.15.11.129:53161)
0809 15:13:35.913213 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:35.913250 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:35.914850 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:35.915462 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:35.915469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:35.915917 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:35.916511 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:35.916528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:35.917479 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:35.917484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:35.919447 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:35.919460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:35.920704 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:35.920717 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:35.920910 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:35.921614 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:35.921641 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:35.924262 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:35.927209 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:35.927217 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:35.927229 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:35.927648 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:35.928039 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:35.928044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:35.928047 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:35.928351 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:35.929022 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:35.929030 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:35.929460 (+ 430us) client_negotiation.cc:769] Sending connection context
0809 15:13:35.929760 (+ 300us) client_negotiation.cc:240] Negotiation successful
0809 15:13:35.930139 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":286,"threads_started":1}
I20260809 15:13:35.932789 18236 heartbeater.cc:347] Connected to a master server at 127.15.11.190:44935
I20260809 15:13:35.933115 18236 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:35.933820 18236 heartbeater.cc:511] Master 127.15.11.190:44935 requested a full tablet report, sending...
I20260809 15:13:35.935782 18026 ts_manager.cc:194] Registered new tserver with Master: e82d77c1339848e198e5678403110d7e (127.15.11.129:41895)
I20260809 15:13:35.937228 18026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:53161
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:13:35.965201 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:44935
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:36.289149 18242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:36.289415 18242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:36.289500 18242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:36.296320 18242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:36.296414 18242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:36.296523 18242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:13:36.305519 18242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:44935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:36.306646 18242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:36.307824 18242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:36.320364 18247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:36.320518 18248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:36.320478 18250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:36.321622 18242 server_base.cc:1034] running on GCE node
I20260809 15:13:36.322127 18242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:36.322955 18242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:36.324204 18242 hybrid_clock.cc:648] HybridClock initialized: now 1786288416324139 us; error 54 us; skew 500 ppm
I20260809 15:13:36.324515 18242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:36 dist-test-slave-8w9v krb5kdc[17978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:36.327924 18242 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:13:36.329402 18242 webserver.cc:460] Webserver started at http://127.15.11.130:45351/ using document root <none> and password file <none>
I20260809 15:13:36.330065 18242 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:36.330199 18242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:36.330518 18242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:36.333361 18242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "fd29970aa1974c1ebab1d6af2a543688"
format_stamp: "Formatted at 2026-08-09 15:13:36 on dist-test-slave-8w9v"
server_key: "a91f4566e95e6835cd1c01487c70d960"
server_key_iv: "1b88af4f59134804b0604800cc5274af"
server_key_version: "encryptionkey@0"
I20260809 15:13:36.334179 18242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "fd29970aa1974c1ebab1d6af2a543688"
format_stamp: "Formatted at 2026-08-09 15:13:36 on dist-test-slave-8w9v"
server_key: "a91f4566e95e6835cd1c01487c70d960"
server_key_iv: "1b88af4f59134804b0604800cc5274af"
server_key_version: "encryptionkey@0"
I20260809 15:13:36.339056 18242 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 15:13:36.342841 18257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:36.344080 18242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:13:36.344306 18242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "fd29970aa1974c1ebab1d6af2a543688"
format_stamp: "Formatted at 2026-08-09 15:13:36 on dist-test-slave-8w9v"
server_key: "a91f4566e95e6835cd1c01487c70d960"
server_key_iv: "1b88af4f59134804b0604800cc5274af"
server_key_version: "encryptionkey@0"
I20260809 15:13:36.344533 18242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:36.359023 18242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:36.363242 18242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:36.363544 18242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:36.364499 18242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:36.365721 18242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:36.365830 18242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:36.365981 18242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:36.366056 18242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:36.399565 18242 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:35123
I20260809 15:13:36.399612 18380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:35123 every 8 connection(s)
I20260809 15:13:36.400911 18242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:13:36.402437 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 18242
I20260809 15:13:36.402643 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:13:36.403254 15406 external_mini_cluster.cc:1428] Setting key 83356f4cc374421fe7362b62565af34a
Aug 09 15:13:36 dist-test-slave-8w9v krb5kdc[17978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:36.421967 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:36.403536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:45901 (local address 127.15.11.190:44935)
0809 15:13:36.403740 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:36.403749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:36.405127 (+ 1378us) server_negotiation.cc:407] Connection header received
0809 15:13:36.407829 (+ 2702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:36.407835 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:36.407932 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:36.408043 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:36.409870 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:36.410599 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:36.412170 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:36.412621 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:36.412762 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:36.416183 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:36.416188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:36.416192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:36.416224 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:36.418926 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:36.419644 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:36.419651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:36.419659 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:36.419781 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:36.420230 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:36.420235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:36.420239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:36.420509 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:36.420680 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:36.421392 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 15:13:36.421616 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:13:36.422926 18383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:36.404105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:44935 (local address 127.15.11.130:45901)
0809 15:13:36.404967 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:36.405010 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:36.407467 (+ 2457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:36.408427 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:36.408435 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:36.409062 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:36.409682 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:36.409700 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:36.410774 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:36.410779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:36.411999 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:36.412011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:36.412854 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:36.412873 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:36.413101 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:36.413801 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:36.413823 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:36.415878 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:36.419087 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:36.419095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:36.419109 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:36.419518 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:36.419900 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:36.419905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:36.419909 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:36.420095 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:36.420643 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:36.420650 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:36.420996 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 15:13:36.421270 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 15:13:36.421804 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":335,"threads_started":1}
I20260809 15:13:36.424506 18381 heartbeater.cc:347] Connected to a master server at 127.15.11.190:44935
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
I20260809 15:13:36.424808 18381 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:36.425545 18381 heartbeater.cc:511] Master 127.15.11.190:44935 requested a full tablet report, sending...
I20260809 15:13:36.426944 18026 ts_manager.cc:194] Registered new tserver with Master: fd29970aa1974c1ebab1d6af2a543688 (127.15.11.130:35123)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
I20260809 15:13:36.427726 18026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:45901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:13:36.449646 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:44935
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:33897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:36.776000 18386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:36.776347 18386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:36.776527 18386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:36.783524 18386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:36.783617 18386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:36.783818 18386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:13:36.793612 18386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:33897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:44935
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:36.794858 18386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:36.796253 18386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:36.809662 18391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:36.809659 18394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:36.809656 18392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:36.810523 18386 server_base.cc:1034] running on GCE node
I20260809 15:13:36.811071 18386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:36.812312 18386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:36.813510 18386 hybrid_clock.cc:648] HybridClock initialized: now 1786288416813461 us; error 46 us; skew 500 ppm
I20260809 15:13:36.813829 18386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:36 dist-test-slave-8w9v krb5kdc[17978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:36.817189 18386 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:13:36.818715 18386 webserver.cc:460] Webserver started at http://127.15.11.131:34115/ using document root <none> and password file <none>
I20260809 15:13:36.819382 18386 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:36.819530 18386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:36.819839 18386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:36.822753 18386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "a013ac7383cb4469ad0eadbf5806bfc9"
format_stamp: "Formatted at 2026-08-09 15:13:36 on dist-test-slave-8w9v"
server_key: "a082d3456c3c9981bb2d3f1f847e2edf"
server_key_iv: "b6f4b736f43d581d82101b5cbc32319e"
server_key_version: "encryptionkey@0"
I20260809 15:13:36.823546 18386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "a013ac7383cb4469ad0eadbf5806bfc9"
format_stamp: "Formatted at 2026-08-09 15:13:36 on dist-test-slave-8w9v"
server_key: "a082d3456c3c9981bb2d3f1f847e2edf"
server_key_iv: "b6f4b736f43d581d82101b5cbc32319e"
server_key_version: "encryptionkey@0"
I20260809 15:13:36.828424 18386 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260809 15:13:36.831640 18401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:36.832901 18386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:36.833083 18386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "a013ac7383cb4469ad0eadbf5806bfc9"
format_stamp: "Formatted at 2026-08-09 15:13:36 on dist-test-slave-8w9v"
server_key: "a082d3456c3c9981bb2d3f1f847e2edf"
server_key_iv: "b6f4b736f43d581d82101b5cbc32319e"
server_key_version: "encryptionkey@0"
I20260809 15:13:36.833279 18386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:36.848165 18386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:36.852396 18386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:36.852689 18386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:36.853580 18386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:36.854892 18386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:36.854997 18386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:36.855154 18386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:36.855228 18386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:36.888186 18386 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:41701
I20260809 15:13:36.888238 18524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:41701 every 8 connection(s)
I20260809 15:13:36.889474 18386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:13:36.897253 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 18386
I20260809 15:13:36.897452 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:13:36.898092 15406 external_mini_cluster.cc:1428] Setting key 8aa8f96f4616b3ab91071535ae5404f5
Aug 09 15:13:36 dist-test-slave-8w9v krb5kdc[17978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288416, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:36.908996 18238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:36.892018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:46753 (local address 127.15.11.190:44935)
0809 15:13:36.892241 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:36.892246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:36.893326 (+ 1080us) server_negotiation.cc:407] Connection header received
0809 15:13:36.895064 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:36.895068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:36.895163 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:36.895250 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:36.896740 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:36.897520 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:36.899707 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:36.900302 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:36.900426 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:36.903834 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:36.903839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:36.903842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:36.903881 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:36.906208 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:36.906826 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:36.906830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:36.906832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:36.906965 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:36.907334 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:36.907352 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:36.907354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:36.907659 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:36.907804 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:36.908627 (+ 823us) server_negotiation.cc:299] Negotiation successful
0809 15:13:36.908769 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:13:36.910162 18528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:36.892460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:44935 (local address 127.15.11.131:46753)
0809 15:13:36.893166 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:36.893204 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:36.894809 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:36.895459 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:36.895467 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:36.895939 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:36.896595 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:36.896613 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:36.897721 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:36.897727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:36.899555 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:36.899571 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:36.900512 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:36.900517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:36.900708 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:36.901418 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:36.901438 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:36.903653 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:36.906325 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:36.906332 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:36.906342 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:36.906732 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:36.907068 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:36.907071 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:36.907073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:36.907229 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:36.907807 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:36.907815 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:36.908198 (+ 383us) client_negotiation.cc:769] Sending connection context
0809 15:13:36.908507 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 15:13:36.909004 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":272,"threads_started":1}
I20260809 15:13:36.911460 18525 heartbeater.cc:347] Connected to a master server at 127.15.11.190:44935
I20260809 15:13:36.911733 18525 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:36.912643 18525 heartbeater.cc:511] Master 127.15.11.190:44935 requested a full tablet report, sending...
I20260809 15:13:36.914149 18026 ts_manager.cc:194] Registered new tserver with Master: a013ac7383cb4469ad0eadbf5806bfc9 (127.15.11.131:41701)
I20260809 15:13:36.914953 18026 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:46753
I20260809 15:13:36.925423 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:13:36.939867 18236 heartbeater.cc:503] Master 127.15.11.190:44935 was elected leader, sending a full tablet report...
W20260809 15:13:36.961745 18238 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:36.959330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44200 (local address 127.15.11.190:44935)
0809 15:13:36.959571 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:36.959581 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:36.959951 (+ 370us) server_negotiation.cc:407] Connection header received
0809 15:13:36.960154 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:36.960166 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:36.960233 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:36.960316 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:36.961480 (+ 1164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44200: BlockingRecv error: recv got EOF from 127.0.0.1:44200 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:13:36.965473 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18096
W20260809 15:13:36.968317 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18096: Not authorized: Client connection negotiation failed: client connection to 127.15.11.129:41895: server requires authentication, but client does not have Kerberos credentials available
I20260809 15:13:36.968479 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 18096
I20260809 15:13:36.985941 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18242
W20260809 15:13:36.988683 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18242: Not authorized: Client connection negotiation failed: client connection to 127.15.11.130:35123: server requires authentication, but client does not have Kerberos credentials available
I20260809 15:13:36.988859 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 18242
I20260809 15:13:37.007740 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18386
W20260809 15:13:37.010728 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18386: Not authorized: Client connection negotiation failed: client connection to 127.15.11.131:41701: server requires authentication, but client does not have Kerberos credentials available
I20260809 15:13:37.010897 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 18386
I20260809 15:13:37.030254 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 17991
I20260809 15:13:37.031394 18062 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:37.133363 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 17991
2026-08-09T15:13:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3637 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:37 dist-test-slave-8w9v krb5kdc[18544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:37 dist-test-slave-8w9v krb5kdc[18544](info): set up 2 sockets
Aug 09 15:13:37 dist-test-slave-8w9v krb5kdc[18544](info): commencing operation
krb5kdc: starting...
W20260809 15:13:38.267830 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.006s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:38 dist-test-slave-8w9v krb5kdc[18544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:38.477773 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35009
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:35009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:38.804378 18558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:38.804708 18558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:38.804858 18558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:38.811702 18558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:38.811842 18558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:38.811968 18558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:38.812080 18558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:38.812187 18558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:38.821484 18558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:35009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:38.822767 18558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:38.824064 18558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:38.835966 18566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:38.836062 18564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:38.836284 18563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:38.836947 18558 server_base.cc:1034] running on GCE node
I20260809 15:13:38.837622 18558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:38.838941 18558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:38.840185 18558 hybrid_clock.cc:648] HybridClock initialized: now 1786288418840097 us; error 33 us; skew 500 ppm
I20260809 15:13:38.840502 18558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:38 dist-test-slave-8w9v krb5kdc[18544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288418, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:38.848937 18558 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:38.850425 18558 webserver.cc:460] Webserver started at http://127.15.11.190:45193/ using document root <none> and password file <none>
I20260809 15:13:38.851078 18558 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:38.851207 18558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:38.851470 18558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:38.854503 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "6126f46591fd4bdaae469a1681d26ec1"
format_stamp: "Formatted at 2026-08-09 15:13:38 on dist-test-slave-8w9v"
server_key: "ccdf0c256f02c821633bd8ff3cc936d3"
server_key_iv: "aff5132f44a85ff1fd47ae77a577a01b"
server_key_version: "encryptionkey@0"
I20260809 15:13:38.855242 18558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "6126f46591fd4bdaae469a1681d26ec1"
format_stamp: "Formatted at 2026-08-09 15:13:38 on dist-test-slave-8w9v"
server_key: "ccdf0c256f02c821633bd8ff3cc936d3"
server_key_iv: "aff5132f44a85ff1fd47ae77a577a01b"
server_key_version: "encryptionkey@0"
I20260809 15:13:38.860339 18558 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:13:38.863636 18574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:38.865015 18558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:38.865206 18558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "6126f46591fd4bdaae469a1681d26ec1"
format_stamp: "Formatted at 2026-08-09 15:13:38 on dist-test-slave-8w9v"
server_key: "ccdf0c256f02c821633bd8ff3cc936d3"
server_key_iv: "aff5132f44a85ff1fd47ae77a577a01b"
server_key_version: "encryptionkey@0"
I20260809 15:13:38.865409 18558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:38.885715 18558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:38.890043 18558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:38.890321 18558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:38.909123 18558 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:35009
I20260809 15:13:38.909160 18630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:35009 every 8 connection(s)
I20260809 15:13:38.910480 18558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:38.914247 18631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:38.917168 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 18558
I20260809 15:13:38.917433 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:38.918097 15406 external_mini_cluster.cc:1428] Setting key e6f5260f4528e20b4911f2d516e31cf9
I20260809 15:13:38.927369 18631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1: Bootstrap starting.
I20260809 15:13:38.930992 18631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:38.931993 18631 log.cc:824] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:13:38 dist-test-slave-8w9v krb5kdc[18544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:38.934713 18631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1: No bootstrap required, opened a new log
I20260809 15:13:38.941497 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:38.920064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50660 (local address 127.15.11.190:35009)
0809 15:13:38.920964 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:38.920980 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:38.921032 (+ 52us) server_negotiation.cc:407] Connection header received
0809 15:13:38.923194 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:38.923229 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:38.923745 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:38.924199 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:38.925725 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:38.927653 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:38.928962 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:38.929985 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:38.930193 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:38.934281 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:38.934289 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:38.934302 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:38.934336 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:38.937518 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:38.938221 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:38.938227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:38.938233 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:38.938409 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:38.938827 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:38.938831 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:38.938833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:38.939342 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:38.939670 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:38.940141 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 15:13:38.940539 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":349,"threads_started":1}
I20260809 15:13:38.942575 18631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126f46591fd4bdaae469a1681d26ec1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35009 } }
I20260809 15:13:38.942935 18631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:38.943041 18631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6126f46591fd4bdaae469a1681d26ec1, State: Initialized, Role: FOLLOWER
I20260809 15:13:38.943619 18631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126f46591fd4bdaae469a1681d26ec1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35009 } }
I20260809 15:13:38.943902 18631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:38.944155 18631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:38.944375 18631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:38.945492 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:13:38.947293 18631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126f46591fd4bdaae469a1681d26ec1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35009 } }
I20260809 15:13:38.947846 18631 leader_election.cc:302] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6126f46591fd4bdaae469a1681d26ec1; no voters:
I20260809 15:13:38.948550 18631 leader_election.cc:288] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:38.948628 18636 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:38.950820 18631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:38.952440 18636 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [term 1 LEADER]: Becoming Leader. State: Replica: 6126f46591fd4bdaae469a1681d26ec1, State: Running, Role: LEADER
I20260809 15:13:38.952977 18636 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126f46591fd4bdaae469a1681d26ec1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35009 } }
I20260809 15:13:38.955534 18638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6126f46591fd4bdaae469a1681d26ec1. Latest consensus state: current_term: 1 leader_uuid: "6126f46591fd4bdaae469a1681d26ec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126f46591fd4bdaae469a1681d26ec1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35009 } } }
I20260809 15:13:38.955892 18638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:38.956485 18637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6126f46591fd4bdaae469a1681d26ec1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6126f46591fd4bdaae469a1681d26ec1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35009 } } }
I20260809 15:13:38.956835 18637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:38.964398 18643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:38.969588 18643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:38.980500 18643 catalog_manager.cc:1347] Generated new cluster ID: 826188c2707a4770aba4489c2d815b6e
I20260809 15:13:38.980721 18643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:39.001600 18643 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:39.003588 18643 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:39.017915 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:39.006977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50664 (local address 127.15.11.190:35009)
0809 15:13:39.007150 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:39.007156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:39.008765 (+ 1609us) server_negotiation.cc:407] Connection header received
0809 15:13:39.008944 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:39.008950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:39.009034 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:39.009169 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:39.010374 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:39.011532 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:39.013385 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:39.014078 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:39.014221 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:39.015461 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:39.015475 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:39.015480 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:39.016401 (+ 921us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:39.016481 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:39.016518 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:39.016629 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:39.017410 (+ 781us) server_negotiation.cc:299] Negotiation successful
0809 15:13:39.017533 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:13:39.018214 18643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6126f46591fd4bdaae469a1681d26ec1: Generated new TSK 0
I20260809 15:13:39.019160 18643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:13:39.028721 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18558
I20260809 15:13:39.030119 18625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:39.119172 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 18558
2026-08-09T15:13:39Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1980 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:39 dist-test-slave-8w9v krb5kdc[18669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:39 dist-test-slave-8w9v krb5kdc[18669](info): set up 2 sockets
Aug 09 15:13:39 dist-test-slave-8w9v krb5kdc[18669](info): commencing operation
krb5kdc: starting...
W20260809 15:13:40.240211 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.080s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:40 dist-test-slave-8w9v krb5kdc[18669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:40.441802 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:46217
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:44349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:46217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:40.763953 18685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:40.764251 18685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:40.764351 18685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:40.771828 18685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:40.771963 18685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:40.772047 18685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:40.772167 18685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:40.772246 18685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:40.782052 18685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:44349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:46217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:46217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:40.783315 18685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:40.784644 18685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:40.798348 18690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:40.798317 18691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:40.798439 18693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:40.800050 18685 server_base.cc:1034] running on GCE node
I20260809 15:13:40.800918 18685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:40.802314 18685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:40.803527 18685 hybrid_clock.cc:648] HybridClock initialized: now 1786288420803473 us; error 46 us; skew 500 ppm
I20260809 15:13:40.803879 18685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:40 dist-test-slave-8w9v krb5kdc[18669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288420, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:40.807042 18685 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:40.808688 18685 webserver.cc:460] Webserver started at http://127.15.11.190:37763/ using document root <none> and password file <none>
I20260809 15:13:40.809325 18685 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:40.809435 18685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:40.809689 18685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:40.812649 18685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "354d8c5178b24737832de7ae566a2d79"
format_stamp: "Formatted at 2026-08-09 15:13:40 on dist-test-slave-8w9v"
server_key: "25f2f7555b07b1cc90d91c644b7fbc20"
server_key_iv: "667ba1de812efb7df4e49723c9c9507c"
server_key_version: "encryptionkey@0"
I20260809 15:13:40.813340 18685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "354d8c5178b24737832de7ae566a2d79"
format_stamp: "Formatted at 2026-08-09 15:13:40 on dist-test-slave-8w9v"
server_key: "25f2f7555b07b1cc90d91c644b7fbc20"
server_key_iv: "667ba1de812efb7df4e49723c9c9507c"
server_key_version: "encryptionkey@0"
I20260809 15:13:40.818468 18685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:13:40.821784 18700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:40.823089 18685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:13:40.823276 18685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "354d8c5178b24737832de7ae566a2d79"
format_stamp: "Formatted at 2026-08-09 15:13:40 on dist-test-slave-8w9v"
server_key: "25f2f7555b07b1cc90d91c644b7fbc20"
server_key_iv: "667ba1de812efb7df4e49723c9c9507c"
server_key_version: "encryptionkey@0"
I20260809 15:13:40.823482 18685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:40.846850 18685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:40.850829 18685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:40.851128 18685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:40.870129 18685 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:46217
I20260809 15:13:40.870172 18756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:46217 every 8 connection(s)
I20260809 15:13:40.871523 18685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:40.875205 18757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:40.879487 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 18685
I20260809 15:13:40.879710 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:40.880573 15406 external_mini_cluster.cc:1428] Setting key 0fd8dd7f712d9be6baf3364e6155960a
I20260809 15:13:40.887740 18757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79: Bootstrap starting.
I20260809 15:13:40.891464 18757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:40.892647 18757 log.cc:824] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:40.896238 18757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79: No bootstrap required, opened a new log
Aug 09 15:13:40 dist-test-slave-8w9v krb5kdc[18669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:40.906049 18762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:40.882649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35998 (local address 127.15.11.190:46217)
0809 15:13:40.883449 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:40.883460 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:40.883501 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:13:40.886168 (+ 2667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:40.886190 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:40.886562 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:40.886876 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:40.888623 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:40.890078 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:40.891905 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:40.892666 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:40.892866 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:40.897626 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:40.897638 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:40.897661 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:40.897704 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:40.901731 (+ 4027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:40.902362 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:40.902372 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:40.902380 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:40.902584 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:40.902919 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:40.902925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:40.902934 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:40.903626 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:40.903935 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:40.904265 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 15:13:40.904683 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":377,"threads_started":1}
I20260809 15:13:40.907007 18757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354d8c5178b24737832de7ae566a2d79" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46217 } }
I20260809 15:13:40.907428 18757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:40.907542 18757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 354d8c5178b24737832de7ae566a2d79, State: Initialized, Role: FOLLOWER
I20260809 15:13:40.908236 18757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354d8c5178b24737832de7ae566a2d79" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46217 } }
I20260809 15:13:40.908514 18757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:40.908658 18757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:40.908841 18757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:40.910969 18757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354d8c5178b24737832de7ae566a2d79" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46217 } }
I20260809 15:13:40.911453 18757 leader_election.cc:302] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 354d8c5178b24737832de7ae566a2d79; no voters:
I20260809 15:13:40.912102 18757 leader_election.cc:288] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:40.912276 18764 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:40.913025 18764 raft_consensus.cc:686] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [term 1 LEADER]: Becoming Leader. State: Replica: 354d8c5178b24737832de7ae566a2d79, State: Running, Role: LEADER
I20260809 15:13:40.913612 18764 consensus_queue.cc:237] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354d8c5178b24737832de7ae566a2d79" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46217 } }
I20260809 15:13:40.914315 18757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:40.916569 18766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 354d8c5178b24737832de7ae566a2d79. Latest consensus state: current_term: 1 leader_uuid: "354d8c5178b24737832de7ae566a2d79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354d8c5178b24737832de7ae566a2d79" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46217 } } }
I20260809 15:13:40.916935 18766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:40.917629 18765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "354d8c5178b24737832de7ae566a2d79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "354d8c5178b24737832de7ae566a2d79" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46217 } } }
I20260809 15:13:40.918030 18765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:40.919047 18771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:40.923522 18771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:40.931861 18771 catalog_manager.cc:1347] Generated new cluster ID: a1a7fbcd14834f3e87f0b5dd21f39de9
I20260809 15:13:40.932008 18771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:40.941047 18771 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:40.942230 18771 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:40.954563 18771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 354d8c5178b24737832de7ae566a2d79: Generated new TSK 0
I20260809 15:13:40.955292 18771 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:13:41.012712 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46217
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:44349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:41.343122 18787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:41.343433 18787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:41.343624 18787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:41.350730 18787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:41.350873 18787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:41.351063 18787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:41.360705 18787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:44349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:41.362038 18787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:41.363515 18787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:41.376796 18792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:41.376888 18793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:41.376922 18795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:41.377737 18787 server_base.cc:1034] running on GCE node
I20260809 15:13:41.378367 18787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:41.379307 18787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:41.380528 18787 hybrid_clock.cc:648] HybridClock initialized: now 1786288421380481 us; error 33 us; skew 500 ppm
I20260809 15:13:41.380842 18787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:41 dist-test-slave-8w9v krb5kdc[18669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:41.383935 18787 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:13:41.385345 18787 webserver.cc:460] Webserver started at http://127.15.11.129:41093/ using document root <none> and password file <none>
I20260809 15:13:41.386031 18787 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:41.386178 18787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:41.386479 18787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:41.389350 18787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "f013ac94a91b42f99eb0f45675936039"
format_stamp: "Formatted at 2026-08-09 15:13:41 on dist-test-slave-8w9v"
server_key: "6caf18137ef189c72189fe435defa978"
server_key_iv: "bdbbadb5e4225775973d74617c12f549"
server_key_version: "encryptionkey@0"
I20260809 15:13:41.390130 18787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "f013ac94a91b42f99eb0f45675936039"
format_stamp: "Formatted at 2026-08-09 15:13:41 on dist-test-slave-8w9v"
server_key: "6caf18137ef189c72189fe435defa978"
server_key_iv: "bdbbadb5e4225775973d74617c12f549"
server_key_version: "encryptionkey@0"
I20260809 15:13:41.395344 18787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 15:13:41.398775 18803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:41.400028 18787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:13:41.400238 18787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "f013ac94a91b42f99eb0f45675936039"
format_stamp: "Formatted at 2026-08-09 15:13:41 on dist-test-slave-8w9v"
server_key: "6caf18137ef189c72189fe435defa978"
server_key_iv: "bdbbadb5e4225775973d74617c12f549"
server_key_version: "encryptionkey@0"
I20260809 15:13:41.400462 18787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:41.419023 18787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:41.423136 18787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:41.423445 18787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:41.424821 18787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:41.427357 18787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:41.427434 18787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:41.427525 18787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:41.427569 18787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:41.460237 18787 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:42011
I20260809 15:13:41.460279 18924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:42011 every 8 connection(s)
I20260809 15:13:41.461613 18787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:13:41.464473 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 18787
I20260809 15:13:41.464704 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:13:41.465550 15406 external_mini_cluster.cc:1428] Setting key 4685323954dba3ed0ba3d46977c58352
Aug 09 15:13:41 dist-test-slave-8w9v krb5kdc[18669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:41.483068 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:41.464163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:44435 (local address 127.15.11.190:46217)
0809 15:13:41.464684 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:41.464689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:41.465880 (+ 1191us) server_negotiation.cc:407] Connection header received
0809 15:13:41.467976 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:41.467982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:41.468070 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:41.468227 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:41.469673 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:41.470387 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:41.471882 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:41.472362 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:41.472506 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:41.476099 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:41.476108 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:41.476145 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:41.476191 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:41.479780 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:41.480563 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:41.480570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:41.480576 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:41.480718 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:41.481115 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:41.481122 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:41.481127 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:41.481494 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:41.481631 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:41.482295 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 15:13:41.482566 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":260,"threads_started":1}
I20260809 15:13:41.483640 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:41.464891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:46217 (local address 127.15.11.129:44435)
0809 15:13:41.465702 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:41.465748 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:41.467710 (+ 1962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:41.468460 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:41.468467 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:41.468975 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:41.469507 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:41.469524 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:41.470531 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:41.470536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:41.471705 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:41.471718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:41.472599 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:41.472608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:41.472841 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:41.473522 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:41.473548 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:41.475863 (+ 2315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:41.479934 (+ 4071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:41.479941 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:41.479951 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:41.480447 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:41.480830 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:41.480833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:41.480835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:41.480997 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:41.481615 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:41.481620 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:41.481888 (+ 268us) client_negotiation.cc:769] Sending connection context
0809 15:13:41.482121 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 15:13:41.482528 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":316,"threads_started":1}
I20260809 15:13:41.485289 18925 heartbeater.cc:347] Connected to a master server at 127.15.11.190:46217
I20260809 15:13:41.485590 18925 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:41.486334 18925 heartbeater.cc:511] Master 127.15.11.190:46217 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
I20260809 15:13:41.488525 18720 ts_manager.cc:194] Registered new tserver with Master: f013ac94a91b42f99eb0f45675936039 (127.15.11.129:42011)
I20260809 15:13:41.490031 18720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:44435
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:13:41.515550 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46217
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:44349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:41.853957 18931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:41.854206 18931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:41.854297 18931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:41.861378 18931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:41.861477 18931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:41.861680 18931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:13:41.871151 18931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:44349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:41.872464 18931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:41.873721 18931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:41.887040 18936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:41.887061 18937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:41.887625 18939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:41.887830 18931 server_base.cc:1034] running on GCE node
I20260809 15:13:41.888527 18931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:41.889456 18931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:41.890666 18931 hybrid_clock.cc:648] HybridClock initialized: now 1786288421890624 us; error 27 us; skew 500 ppm
I20260809 15:13:41.890947 18931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:41 dist-test-slave-8w9v krb5kdc[18669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:41.894129 18931 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:13:41.895526 18931 webserver.cc:460] Webserver started at http://127.15.11.130:44825/ using document root <none> and password file <none>
I20260809 15:13:41.896224 18931 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:41.896329 18931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:41.896585 18931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:41.899495 18931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "db08a593ed7c4f78adec6d9eb0ec3c69"
format_stamp: "Formatted at 2026-08-09 15:13:41 on dist-test-slave-8w9v"
server_key: "dfda65007f2953d35d0509b9532d25e4"
server_key_iv: "b444c31cd4c101e1e046d69bc67cdd4d"
server_key_version: "encryptionkey@0"
I20260809 15:13:41.900296 18931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "db08a593ed7c4f78adec6d9eb0ec3c69"
format_stamp: "Formatted at 2026-08-09 15:13:41 on dist-test-slave-8w9v"
server_key: "dfda65007f2953d35d0509b9532d25e4"
server_key_iv: "b444c31cd4c101e1e046d69bc67cdd4d"
server_key_version: "encryptionkey@0"
I20260809 15:13:41.905165 18931 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:13:41.908401 18947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:41.909627 18931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:13:41.909749 18931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "db08a593ed7c4f78adec6d9eb0ec3c69"
format_stamp: "Formatted at 2026-08-09 15:13:41 on dist-test-slave-8w9v"
server_key: "dfda65007f2953d35d0509b9532d25e4"
server_key_iv: "b444c31cd4c101e1e046d69bc67cdd4d"
server_key_version: "encryptionkey@0"
I20260809 15:13:41.909893 18931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:41.923813 18931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:41.927889 18931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:41.928220 18931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:41.929353 18931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:41.930707 18931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:41.930807 18931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:41.930903 18931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:41.930946 18931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:41.964635 18931 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:32805
I20260809 15:13:41.964730 19067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:32805 every 8 connection(s)
I20260809 15:13:41.965936 18931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:13:41.973871 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 18931
I20260809 15:13:41.974052 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:13:41.974651 15406 external_mini_cluster.cc:1428] Setting key f5f04f2a550379f9772f239379070fce
Aug 09 15:13:41 dist-test-slave-8w9v krb5kdc[18669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288421, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:41.987049 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:41.968432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:42459 (local address 127.15.11.190:46217)
0809 15:13:41.968617 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:41.968621 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:41.969779 (+ 1158us) server_negotiation.cc:407] Connection header received
0809 15:13:41.971607 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:41.971613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:41.971702 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:41.971812 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:41.973329 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:41.974406 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:41.976262 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:41.976925 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:41.977024 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:41.981174 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:41.981178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:41.981181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:41.981211 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:41.983797 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:41.984731 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:41.984736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:41.984741 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:41.984872 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:41.985321 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:41.985325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:41.985327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:41.985591 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:41.985764 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:41.986480 (+ 716us) server_negotiation.cc:299] Negotiation successful
0809 15:13:41.986705 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:13:41.987988 19071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:41.968922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:46217 (local address 127.15.11.130:42459)
0809 15:13:41.969624 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:41.969661 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:41.971326 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:41.972057 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:41.972064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:41.972614 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:41.973162 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:41.973179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:41.974555 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:41.974560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:41.976055 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:41.976069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:41.977172 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:41.977179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:41.977467 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:41.978172 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:41.978207 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:41.980939 (+ 2732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:41.983957 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:41.983965 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:41.983980 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:41.984605 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:41.984995 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:41.985000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:41.985004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:41.985195 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:41.985737 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:41.985745 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:41.986102 (+ 357us) client_negotiation.cc:769] Sending connection context
0809 15:13:41.986366 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 15:13:41.986885 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":272,"threads_started":1}
I20260809 15:13:41.989575 19069 heartbeater.cc:347] Connected to a master server at 127.15.11.190:46217
I20260809 15:13:41.989863 19069 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:41.990620 19069 heartbeater.cc:511] Master 127.15.11.190:46217 requested a full tablet report, sending...
I20260809 15:13:41.992127 18720 ts_manager.cc:194] Registered new tserver with Master: db08a593ed7c4f78adec6d9eb0ec3c69 (127.15.11.130:32805)
I20260809 15:13:41.993245 18720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:42459
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:13:42.023644 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46217
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:44349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:42.351693 19074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:42.352002 19074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:42.352221 19074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:42.359308 19074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:42.359462 19074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:42.359635 19074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:13:42.369335 19074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:44349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:42.370743 19074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:42.372109 19074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:42.385257 19082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:42.385275 19079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:42.385349 19080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:42.386790 19074 server_base.cc:1034] running on GCE node
I20260809 15:13:42.387301 19074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:42.388190 19074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:42.389407 19074 hybrid_clock.cc:648] HybridClock initialized: now 1786288422389368 us; error 31 us; skew 500 ppm
I20260809 15:13:42.389722 19074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[18669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288422, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:42.392880 19074 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:13:42.394275 19074 webserver.cc:460] Webserver started at http://127.15.11.131:44721/ using document root <none> and password file <none>
I20260809 15:13:42.394974 19074 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:42.395112 19074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:42.395448 19074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:42.398383 19074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "4e1fe384a10a42ac9779c7f36713b33b"
format_stamp: "Formatted at 2026-08-09 15:13:42 on dist-test-slave-8w9v"
server_key: "6cf5eff6b9938bab2904cb0aa41ce5d2"
server_key_iv: "ca29c0546798ab3db53fd9415917b375"
server_key_version: "encryptionkey@0"
I20260809 15:13:42.399173 19074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "4e1fe384a10a42ac9779c7f36713b33b"
format_stamp: "Formatted at 2026-08-09 15:13:42 on dist-test-slave-8w9v"
server_key: "6cf5eff6b9938bab2904cb0aa41ce5d2"
server_key_iv: "ca29c0546798ab3db53fd9415917b375"
server_key_version: "encryptionkey@0"
I20260809 15:13:42.404579 19074 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 15:13:42.408289 19089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:42.409648 19074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:42.409835 19074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "4e1fe384a10a42ac9779c7f36713b33b"
format_stamp: "Formatted at 2026-08-09 15:13:42 on dist-test-slave-8w9v"
server_key: "6cf5eff6b9938bab2904cb0aa41ce5d2"
server_key_iv: "ca29c0546798ab3db53fd9415917b375"
server_key_version: "encryptionkey@0"
I20260809 15:13:42.410050 19074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:42.425371 19074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:42.430424 19074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:42.430760 19074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:42.431732 19074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:42.433128 19074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:42.433202 19074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:42.433285 19074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:42.433333 19074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:42.471076 19074 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:42849
I20260809 15:13:42.471113 19211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:42849 every 8 connection(s)
I20260809 15:13:42.472417 19074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:13:42.473362 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 19074
I20260809 15:13:42.473536 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:13:42.474023 15406 external_mini_cluster.cc:1428] Setting key 46dfc5dc93b9a181032ee1208e36cff8
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[18669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288422, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:42.493036 18925 heartbeater.cc:503] Master 127.15.11.190:46217 was elected leader, sending a full tablet report...
I20260809 15:13:42.493126 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:42.475251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:37249 (local address 127.15.11.190:46217)
0809 15:13:42.475478 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:42.475484 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:42.476904 (+ 1420us) server_negotiation.cc:407] Connection header received
0809 15:13:42.478730 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:42.478734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:42.478810 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:42.478889 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:42.480395 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.481155 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.483263 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.483640 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.483737 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:42.486995 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:42.487000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:42.487003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:42.487034 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:42.490380 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.491034 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.491038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.491040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.491164 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.491545 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.491548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.491550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.491796 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:42.491903 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:42.492578 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 15:13:42.492799 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:13:42.494068 19214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:42.475760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:46217 (local address 127.15.11.131:37249)
0809 15:13:42.476714 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:42.476766 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:42.478499 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:42.479111 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:42.479119 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:42.479641 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:42.480255 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:42.480273 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.481300 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.481304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:42.483137 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:42.483149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.483815 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.483820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:42.484011 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:42.484655 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:42.484676 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:42.486798 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:42.490498 (+ 3700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:42.490505 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:42.490515 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:42.490927 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:42.491256 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:42.491259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:42.491261 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:42.491446 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:42.491905 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:42.491910 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:42.492257 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 15:13:42.492468 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 15:13:42.492918 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":378,"threads_started":1}
I20260809 15:13:42.495451 19212 heartbeater.cc:347] Connected to a master server at 127.15.11.190:46217
I20260809 15:13:42.495739 19212 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:42.496677 19212 heartbeater.cc:511] Master 127.15.11.190:46217 requested a full tablet report, sending...
I20260809 15:13:42.498173 18720 ts_manager.cc:194] Registered new tserver with Master: 4e1fe384a10a42ac9779c7f36713b33b (127.15.11.131:42849)
I20260809 15:13:42.498931 18720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:37249
I20260809 15:13:42.502121 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[18669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288422, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[18669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288422, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:42.554242 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:42.540980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36010 (local address 127.15.11.190:46217)
0809 15:13:42.541225 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:42.541235 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:42.541648 (+ 413us) server_negotiation.cc:407] Connection header received
0809 15:13:42.541814 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:42.541818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:42.541892 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:42.541978 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:42.543271 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.543952 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.545148 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.545533 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.545620 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:42.549492 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:42.549496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:42.549500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:42.549531 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:42.551915 (+ 2384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.552538 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.552542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.552544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.552673 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.553028 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.553031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.553033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.553326 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:42.553508 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:42.553746 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 15:13:42.553910 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 15:13:42.554843 18720 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36010
I20260809 15:13:42.556425 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18787
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[18669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288422, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:42.571242 19223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:42.557422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.15.11.129:42011)
0809 15:13:42.558048 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:42.558054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:42.558079 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:13:42.558134 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:42.558140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:42.558347 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:42.558469 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:42.559608 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.560702 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.561858 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.562407 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.562502 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:42.566235 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:42.566246 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:42.566259 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:42.566292 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:42.569000 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.569604 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.569611 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.569613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.569725 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.570055 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.570058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.570060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.570364 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:42.570580 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:42.570778 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 15:13:42.570972 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":301,"threads_started":1}
W20260809 15:13:42.572480 18920 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58530
W20260809 15:13:42.573591 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18787: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:13:42.573820 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 18787
I20260809 15:13:42.593525 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18931
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[18669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288422, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:13:42.608955 19224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:42.594335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.15.11.130:32805)
0809 15:13:42.594908 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:42.594914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:42.594939 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:13:42.594995 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:42.595001 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:42.595199 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:42.595327 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:42.596573 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.597592 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.598808 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.599341 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.599448 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:42.603461 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:42.603473 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:42.603486 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:42.603517 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:42.606379 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.607060 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.607066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.607069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.607189 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.607491 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.607494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.607496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.607800 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:42.608038 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:42.608478 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 15:13:42.608672 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":313,"threads_started":1}
W20260809 15:13:42.610216 19063 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59324
W20260809 15:13:42.611307 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18931: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:13:42.611478 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 18931
I20260809 15:13:42.631858 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 19074
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[18669](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288422, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:13:42.647796 19225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:42.632954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49900 (local address 127.15.11.131:42849)
0809 15:13:42.633603 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:42.633610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:42.633639 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:13:42.633699 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:42.633705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:42.633924 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:42.634055 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:42.635417 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.636557 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.637723 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:42.638275 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:42.638372 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:42.642628 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:42.642638 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:42.642652 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:42.642681 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:42.645377 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.645995 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.646001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.646004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.646119 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:42.646408 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:42.646411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:42.646413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:42.646734 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:42.646974 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:42.647203 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 15:13:42.647396 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":328,"threads_started":1}
W20260809 15:13:42.649186 19207 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49900
W20260809 15:13:42.650213 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 19074: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:13:42.650360 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 19074
I20260809 15:13:42.671281 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 18685
I20260809 15:13:42.672529 18752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:42.776523 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 18685
2026-08-09T15:13:42Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3664 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[19229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[19229](info): set up 2 sockets
Aug 09 15:13:42 dist-test-slave-8w9v krb5kdc[19229](info): commencing operation
krb5kdc: starting...
W20260809 15:13:43.904840 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:44 dist-test-slave-8w9v krb5kdc[19229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:44.115480 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39297
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:39297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:44.446130 19243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:44.446473 19243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:44.446627 19243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:44.453645 19243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:44.453778 19243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:44.453883 19243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:44.453989 19243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:44.454100 19243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:44.464161 19243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:39297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:44.465512 19243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:44.466869 19243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:44.478645 19249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:44.478703 19252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:44.478729 19250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:44.479374 19243 server_base.cc:1034] running on GCE node
I20260809 15:13:44.480036 19243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:44.481324 19243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:44.482580 19243 hybrid_clock.cc:648] HybridClock initialized: now 1786288424482531 us; error 57 us; skew 500 ppm
I20260809 15:13:44.482924 19243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:44 dist-test-slave-8w9v krb5kdc[19229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288424, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:44.490635 19243 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:44.492013 19243 webserver.cc:460] Webserver started at http://127.15.11.190:46811/ using document root <none> and password file <none>
I20260809 15:13:44.492703 19243 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:44.492844 19243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:44.493156 19243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:44.496039 19243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a"
format_stamp: "Formatted at 2026-08-09 15:13:44 on dist-test-slave-8w9v"
server_key: "7e0a942ffdb1d00cbf4b1f81253474d5"
server_key_iv: "0a1e467161c4ff40d48a690feeebdbff"
server_key_version: "encryptionkey@0"
I20260809 15:13:44.496852 19243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a"
format_stamp: "Formatted at 2026-08-09 15:13:44 on dist-test-slave-8w9v"
server_key: "7e0a942ffdb1d00cbf4b1f81253474d5"
server_key_iv: "0a1e467161c4ff40d48a690feeebdbff"
server_key_version: "encryptionkey@0"
I20260809 15:13:44.501857 19243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:13:44.505144 19259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:44.506306 19243 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260809 15:13:44.506484 19243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a"
format_stamp: "Formatted at 2026-08-09 15:13:44 on dist-test-slave-8w9v"
server_key: "7e0a942ffdb1d00cbf4b1f81253474d5"
server_key_iv: "0a1e467161c4ff40d48a690feeebdbff"
server_key_version: "encryptionkey@0"
I20260809 15:13:44.506666 19243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:44.521683 19243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:44.525956 19243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:44.526224 19243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:44.544005 19243 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:39297
I20260809 15:13:44.544027 19314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:39297 every 8 connection(s)
I20260809 15:13:44.545300 19243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:44.546103 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 19243
I20260809 15:13:44.546384 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:44.546994 15406 external_mini_cluster.cc:1428] Setting key 5420be05d79bfa26956135ab0f1e5eff
I20260809 15:13:44.549604 19315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:13:44 dist-test-slave-8w9v krb5kdc[19229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:44.562337 19315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a: Bootstrap starting.
I20260809 15:13:44.565739 19315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:44.566771 19315 log.cc:824] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:44.569361 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:44.548888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52156 (local address 127.15.11.190:39297)
0809 15:13:44.549707 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:44.549721 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:44.549770 (+ 49us) server_negotiation.cc:407] Connection header received
0809 15:13:44.551303 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:44.551322 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:44.551654 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:44.551962 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:44.554000 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:44.555283 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:44.557117 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:44.557744 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:44.557876 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:44.562339 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:44.562346 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:44.562357 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:44.562397 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:44.565458 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:44.566210 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:44.566217 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:44.566222 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:44.566369 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:44.566765 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:44.566768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:44.566770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:44.567365 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:44.567690 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:44.568031 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 15:13:44.568453 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":371,"threads_started":1}
I20260809 15:13:44.569708 19315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a: No bootstrap required, opened a new log
I20260809 15:13:44.576930 19315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39297 } }
I20260809 15:13:44.577296 19315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:44.577411 19315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1edbb6433b4fed9fd6bc475a8cda0a, State: Initialized, Role: FOLLOWER
I20260809 15:13:44.578004 19315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39297 } }
I20260809 15:13:44.578243 19315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:44.578419 19315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:44.578629 19315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:44.580739 19315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39297 } }
I20260809 15:13:44.581236 19315 leader_election.cc:302] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d1edbb6433b4fed9fd6bc475a8cda0a; no voters:
I20260809 15:13:44.581832 19315 leader_election.cc:288] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:44.581962 19320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:44.583189 19320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [term 1 LEADER]: Becoming Leader. State: Replica: 8d1edbb6433b4fed9fd6bc475a8cda0a, State: Running, Role: LEADER
I20260809 15:13:44.583925 19315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:44.583971 19320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39297 } }
I20260809 15:13:44.587149 19321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39297 } } }
I20260809 15:13:44.587106 19322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d1edbb6433b4fed9fd6bc475a8cda0a. Latest consensus state: current_term: 1 leader_uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d1edbb6433b4fed9fd6bc475a8cda0a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39297 } } }
I20260809 15:13:44.587657 19321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:44.587657 19322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:44.588608 19330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:44.596405 19330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:44.609558 19330 catalog_manager.cc:1347] Generated new cluster ID: 9d6dd331ada34376a195be9e71ace303
I20260809 15:13:44.609663 19330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:44.641175 19330 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:44.642546 19330 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:44.649592 19330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d1edbb6433b4fed9fd6bc475a8cda0a: Generated new TSK 0
I20260809 15:13:44.650498 19330 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:13:44.717751 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39297
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:45.068635 19344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:45.068895 19344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:45.069000 19344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:45.076191 19344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:45.076298 19344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:45.076406 19344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:45.086519 19344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:45.087782 19344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:45.089174 19344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:45.102707 19349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:45.102813 19351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:45.102764 19353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:45.103474 19344 server_base.cc:1034] running on GCE node
I20260809 15:13:45.104056 19344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:45.104971 19344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:45.106194 19344 hybrid_clock.cc:648] HybridClock initialized: now 1786288425106131 us; error 58 us; skew 500 ppm
I20260809 15:13:45.106506 19344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:45 dist-test-slave-8w9v krb5kdc[19229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:45.109844 19344 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:13:45.111402 19344 webserver.cc:460] Webserver started at http://127.15.11.129:37059/ using document root <none> and password file <none>
I20260809 15:13:45.112087 19344 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:45.112257 19344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:45.112581 19344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:45.115427 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "06d618d572c54d9290779c4e861d6e1e"
format_stamp: "Formatted at 2026-08-09 15:13:45 on dist-test-slave-8w9v"
server_key: "2452a9bbc25624235de59918497b27c4"
server_key_iv: "8d7401b6aec1275eb9fd2e68ca3a990b"
server_key_version: "encryptionkey@0"
I20260809 15:13:45.116309 19344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "06d618d572c54d9290779c4e861d6e1e"
format_stamp: "Formatted at 2026-08-09 15:13:45 on dist-test-slave-8w9v"
server_key: "2452a9bbc25624235de59918497b27c4"
server_key_iv: "8d7401b6aec1275eb9fd2e68ca3a990b"
server_key_version: "encryptionkey@0"
I20260809 15:13:45.121238 19344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:13:45.124702 19360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:45.125960 19344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 15:13:45.126137 19344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "06d618d572c54d9290779c4e861d6e1e"
format_stamp: "Formatted at 2026-08-09 15:13:45 on dist-test-slave-8w9v"
server_key: "2452a9bbc25624235de59918497b27c4"
server_key_iv: "8d7401b6aec1275eb9fd2e68ca3a990b"
server_key_version: "encryptionkey@0"
I20260809 15:13:45.126350 19344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:45.149567 19344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:45.153786 19344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:45.154083 19344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:45.154959 19344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:45.156404 19344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:45.156477 19344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:45.156567 19344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:45.156615 19344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:45.190266 19344 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:41859
I20260809 15:13:45.190349 19481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:41859 every 8 connection(s)
I20260809 15:13:45.191622 19344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:13:45.192914 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 19344
I20260809 15:13:45.193138 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:13:45.194042 15406 external_mini_cluster.cc:1428] Setting key 0e788391e87c0e0977cfb33263510dee
Aug 09 15:13:45 dist-test-slave-8w9v krb5kdc[19229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:45.212345 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:45.194565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:57013 (local address 127.15.11.190:39297)
0809 15:13:45.195060 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:45.195065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:45.196216 (+ 1151us) server_negotiation.cc:407] Connection header received
0809 15:13:45.198037 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:45.198043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:45.198127 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:45.198239 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:45.199797 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:45.200567 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:45.202065 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:45.202490 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:45.202629 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:45.206487 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:45.206492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:45.206496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:45.206528 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:45.209091 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:45.209812 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:45.209819 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:45.209822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:45.209966 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:45.210495 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:45.210500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:45.210505 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:45.210771 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:45.210938 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:45.211713 (+ 775us) server_negotiation.cc:299] Negotiation successful
0809 15:13:45.211952 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":262,"threads_started":1}
I20260809 15:13:45.213346 19485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:45.195177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39297 (local address 127.15.11.129:57013)
0809 15:13:45.196034 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:45.196084 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:45.197769 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:45.198479 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:45.198487 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:45.199074 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:45.199630 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:45.199647 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:45.200722 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:45.200726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:45.201919 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:45.201931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:45.202727 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:45.202735 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:45.203088 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:45.203866 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:45.203898 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:45.206274 (+ 2376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:45.209228 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:45.209236 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:45.209251 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:45.209689 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:45.210100 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:45.210104 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:45.210108 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:45.210377 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:45.210908 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:45.210916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:45.211258 (+ 342us) client_negotiation.cc:769] Sending connection context
0809 15:13:45.211607 (+ 349us) client_negotiation.cc:240] Negotiation successful
0809 15:13:45.212127 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":308,"threads_started":1}
I20260809 15:13:45.215042 19482 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39297
I20260809 15:13:45.215374 19482 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
I20260809 15:13:45.216203 19482 heartbeater.cc:511] Master 127.15.11.190:39297 requested a full tablet report, sending...
I20260809 15:13:45.218353 19278 ts_manager.cc:194] Registered new tserver with Master: 06d618d572c54d9290779c4e861d6e1e (127.15.11.129:41859)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
I20260809 15:13:45.219864 19278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:57013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:13:45.243316 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39297
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:45.575459 19489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:45.575719 19489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:45.575811 19489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:45.582736 19489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:45.582832 19489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:45.583037 19489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:13:45.592720 19489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:45.593931 19489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:45.595237 19489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:45.608227 19494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:45.608315 19495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:45.608744 19497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:45.609169 19489 server_base.cc:1034] running on GCE node
I20260809 15:13:45.609787 19489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:45.610733 19489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:45.611954 19489 hybrid_clock.cc:648] HybridClock initialized: now 1786288425611911 us; error 56 us; skew 500 ppm
I20260809 15:13:45.612313 19489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:45 dist-test-slave-8w9v krb5kdc[19229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:45.615789 19489 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:13:45.617184 19489 webserver.cc:460] Webserver started at http://127.15.11.130:40211/ using document root <none> and password file <none>
I20260809 15:13:45.617779 19489 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:45.617871 19489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:45.618108 19489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:45.620816 19489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "eef6a950ff434857bac2b7ff0d3033ea"
format_stamp: "Formatted at 2026-08-09 15:13:45 on dist-test-slave-8w9v"
server_key: "c625566392aac4829af18a684346da2e"
server_key_iv: "57affda7ba49f9d98853c5b3edc917f1"
server_key_version: "encryptionkey@0"
I20260809 15:13:45.621518 19489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "eef6a950ff434857bac2b7ff0d3033ea"
format_stamp: "Formatted at 2026-08-09 15:13:45 on dist-test-slave-8w9v"
server_key: "c625566392aac4829af18a684346da2e"
server_key_iv: "57affda7ba49f9d98853c5b3edc917f1"
server_key_version: "encryptionkey@0"
I20260809 15:13:45.626317 19489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:13:45.629812 19504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:45.631194 19489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 15:13:45.631362 19489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "eef6a950ff434857bac2b7ff0d3033ea"
format_stamp: "Formatted at 2026-08-09 15:13:45 on dist-test-slave-8w9v"
server_key: "c625566392aac4829af18a684346da2e"
server_key_iv: "57affda7ba49f9d98853c5b3edc917f1"
server_key_version: "encryptionkey@0"
I20260809 15:13:45.631561 19489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:45.646714 19489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:45.650481 19489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:45.650766 19489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:45.651683 19489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:45.653052 19489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:45.653153 19489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:45.653306 19489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:45.653379 19489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:45.685945 19489 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:35203
I20260809 15:13:45.685992 19623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:35203 every 8 connection(s)
I20260809 15:13:45.687224 19489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:13:45.693005 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 19489
I20260809 15:13:45.693212 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:13:45.693784 15406 external_mini_cluster.cc:1428] Setting key ec0f7c49b880eea8b0dba042696cf004
Aug 09 15:13:45 dist-test-slave-8w9v krb5kdc[19229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288425, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:45.708662 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:45.690114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:44863 (local address 127.15.11.190:39297)
0809 15:13:45.690300 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:45.690305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:45.691436 (+ 1131us) server_negotiation.cc:407] Connection header received
0809 15:13:45.693458 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:45.693464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:45.693556 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:45.693671 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:45.695236 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:45.695932 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:45.697538 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:45.697987 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:45.698117 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:45.702128 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:45.702132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:45.702135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:45.702165 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:45.705458 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:45.706147 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:45.706155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:45.706162 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:45.706319 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:45.706720 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:45.706728 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:45.706733 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:45.707122 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:45.707287 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:45.707907 (+ 620us) server_negotiation.cc:299] Negotiation successful
0809 15:13:45.708209 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:13:45.709223 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:45.690527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39297 (local address 127.15.11.130:44863)
0809 15:13:45.691258 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:45.691295 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:45.693075 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:45.693883 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:45.693906 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:45.694395 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:45.695065 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:45.695083 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:45.696171 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:45.696175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:45.697391 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:45.697404 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:45.698222 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:45.698230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:45.698446 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:45.699115 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:45.699141 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:45.701919 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:45.705590 (+ 3671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:45.705596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:45.705607 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:45.706017 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:45.706428 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:45.706432 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:45.706434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:45.706603 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:45.707254 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:45.707259 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:45.707555 (+ 296us) client_negotiation.cc:769] Sending connection context
0809 15:13:45.707758 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 15:13:45.708173 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":273,"threads_started":1}
I20260809 15:13:45.710646 19624 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39297
I20260809 15:13:45.710953 19624 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:45.711836 19624 heartbeater.cc:511] Master 127.15.11.190:39297 requested a full tablet report, sending...
I20260809 15:13:45.713574 19278 ts_manager.cc:194] Registered new tserver with Master: eef6a950ff434857bac2b7ff0d3033ea (127.15.11.130:35203)
I20260809 15:13:45.714883 19278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:44863
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:13:45.745043 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39297
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:46.073532 19629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:46.073809 19629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:46.074045 19629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:46.081110 19629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:46.081203 19629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:46.081400 19629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:13:46.091010 19629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39297
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:46.092377 19629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:46.093729 19629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:46.107313 19634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:46.107216 19635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:46.107260 19637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:46.108043 19629 server_base.cc:1034] running on GCE node
I20260809 15:13:46.108632 19629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:46.109525 19629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:46.110788 19629 hybrid_clock.cc:648] HybridClock initialized: now 1786288426110713 us; error 61 us; skew 500 ppm
I20260809 15:13:46.111097 19629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288426, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:46.114256 19629 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:13:46.115587 19629 webserver.cc:460] Webserver started at http://127.15.11.131:34065/ using document root <none> and password file <none>
I20260809 15:13:46.116312 19629 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:46.116456 19629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:46.116757 19629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:46.119493 19629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "100404aa5f7946ff96d59d891b0604ea"
format_stamp: "Formatted at 2026-08-09 15:13:46 on dist-test-slave-8w9v"
server_key: "8d2fa305f8d884559ef187df3e5f295d"
server_key_iv: "6bf22fb2580903196703a4f90d1e1a84"
server_key_version: "encryptionkey@0"
I20260809 15:13:46.120322 19629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "100404aa5f7946ff96d59d891b0604ea"
format_stamp: "Formatted at 2026-08-09 15:13:46 on dist-test-slave-8w9v"
server_key: "8d2fa305f8d884559ef187df3e5f295d"
server_key_iv: "6bf22fb2580903196703a4f90d1e1a84"
server_key_version: "encryptionkey@0"
I20260809 15:13:46.125496 19629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:13:46.128934 19645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:46.130373 19629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:46.130561 19629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "100404aa5f7946ff96d59d891b0604ea"
format_stamp: "Formatted at 2026-08-09 15:13:46 on dist-test-slave-8w9v"
server_key: "8d2fa305f8d884559ef187df3e5f295d"
server_key_iv: "6bf22fb2580903196703a4f90d1e1a84"
server_key_version: "encryptionkey@0"
I20260809 15:13:46.130766 19629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:46.148988 19629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:46.152537 19629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:46.152786 19629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:46.153640 19629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:46.154911 19629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:46.154982 19629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:46.155068 19629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:46.155128 19629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:46.188230 19629 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:39045
I20260809 15:13:46.188320 19764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:39045 every 8 connection(s)
I20260809 15:13:46.189636 19629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:13:46.195719 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 19629
I20260809 15:13:46.195942 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:13:46.196482 15406 external_mini_cluster.cc:1428] Setting key a705892fd2f2ae7fb4dbadf514750377
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288426, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:46.209369 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:46.192377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:45391 (local address 127.15.11.190:39297)
0809 15:13:46.192614 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:46.192621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:46.193812 (+ 1191us) server_negotiation.cc:407] Connection header received
0809 15:13:46.195625 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:46.195630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:46.195755 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:46.195899 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:46.197647 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.198429 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.200147 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.200572 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.200732 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:46.204258 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:46.204262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:46.204265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:46.204296 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:46.206815 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.207443 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.207447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.207449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.207559 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.207925 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.207928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.207930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.208239 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:46.208346 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:46.208926 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 15:13:46.209077 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 15:13:46.210427 19767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:46.192783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39297 (local address 127.15.11.131:45391)
0809 15:13:46.193643 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:46.193684 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:46.195336 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:46.196209 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:46.196225 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:46.196803 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:46.197476 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:46.197494 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.198580 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.198586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:46.199983 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:46.199996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.200821 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.200831 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:46.201090 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:46.201785 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:46.201811 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:46.204026 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:46.206941 (+ 2915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:46.206947 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:46.206958 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:46.207337 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:46.207651 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:46.207654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:46.207656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:46.207822 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:46.208340 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:46.208345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:46.208616 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 15:13:46.208822 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 15:13:46.209166 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":355,"threads_started":1}
I20260809 15:13:46.211987 19765 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39297
I20260809 15:13:46.212316 19765 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:46.213021 19765 heartbeater.cc:511] Master 127.15.11.190:39297 requested a full tablet report, sending...
I20260809 15:13:46.214329 19278 ts_manager.cc:194] Registered new tserver with Master: 100404aa5f7946ff96d59d891b0604ea (127.15.11.131:39045)
I20260809 15:13:46.215055 19278 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:45391
I20260809 15:13:46.223562 19482 heartbeater.cc:503] Master 127.15.11.190:39297 was elected leader, sending a full tablet report...
I20260809 15:13:46.224030 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:46.277047 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:46.261870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.15.11.129:41859)
0809 15:13:46.262586 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:46.262596 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:46.262629 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:13:46.262866 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:46.262874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:46.263099 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:46.263271 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:46.264844 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.265836 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.267107 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.267670 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.267811 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:46.271747 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:46.271757 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:46.271770 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:46.271802 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:46.274474 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.275062 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.275068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.275070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.275186 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.275506 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.275509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.275511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.275817 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:46.276058 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:46.276410 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 15:13:46.276683 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":303,"threads_started":1}
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:46.294623 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:46.281579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52170 (local address 127.15.11.190:39297)
0809 15:13:46.281785 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:46.281792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:46.281811 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:13:46.281873 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:46.281876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:46.281940 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:46.282043 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:46.283262 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.283955 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.285295 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.285791 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.285931 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:46.289601 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:46.289606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:46.289609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:46.289641 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:46.291993 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.292631 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.292637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.292641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.292777 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.293149 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.293154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.293157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.293501 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:46.293748 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:46.294060 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 15:13:46.294331 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 15:13:46.295161 19278 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52170
I20260809 15:13:46.296198 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 19344
I20260809 15:13:46.308382 19772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:46.297009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41038 (local address 127.15.11.129:41859)
0809 15:13:46.297220 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:46.297226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:46.297454 (+ 228us) server_negotiation.cc:407] Connection header received
0809 15:13:46.297632 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:46.297637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:46.297735 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:46.297831 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:46.299000 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.299682 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.300937 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.301445 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.301581 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:46.303623 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:46.303629 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:46.303634 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:46.303671 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:46.306009 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.306616 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.306622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.306625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.306742 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.307098 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.307103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.307107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.307333 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:46.307571 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:46.307855 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 15:13:46.308071 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:13:46.308905 19477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:46.390501 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 19344
I20260809 15:13:46.411314 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 19489
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:13:46.426877 19777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:46.412328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.15.11.130:35203)
0809 15:13:46.412890 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:46.412898 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:46.412940 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:13:46.413007 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:46.413013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:46.413213 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:46.413341 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:46.414512 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.415463 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.416860 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.417378 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.417474 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:46.421464 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:46.421476 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:46.421489 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:46.421520 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:46.424476 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.425083 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.425090 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.425092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.425221 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.425527 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.425530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.425532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.425847 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:46.426072 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:46.426287 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 15:13:46.426520 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":312,"threads_started":1}
I20260809 15:13:46.428112 19619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:46.511154 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 19489
I20260809 15:13:46.532689 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 19629
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288426, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:13:46.549145 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:46.533705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.15.11.131:39045)
0809 15:13:46.534303 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:46.534310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:46.534341 (+ 31us) server_negotiation.cc:407] Connection header received
0809 15:13:46.534416 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:46.534422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:46.534706 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:46.534832 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:46.536198 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.537252 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.538520 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:46.539065 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:46.539160 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:46.543437 (+ 4277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:46.543447 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:46.543462 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:46.543494 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:46.546328 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.547033 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.547040 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.547042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.547162 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:46.547544 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:46.547550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:46.547553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:46.547886 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:46.548270 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:46.548519 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 15:13:46.548796 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":284,"threads_started":1}
I20260809 15:13:46.550525 19760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:46.633118 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 19629
I20260809 15:13:46.653546 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 19243
I20260809 15:13:46.654829 19310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:46.763046 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 19243
2026-08-09T15:13:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3985 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19785](info): set up 2 sockets
Aug 09 15:13:46 dist-test-slave-8w9v krb5kdc[19785](info): commencing operation
krb5kdc: starting...
W20260809 15:13:47.896692 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:48 dist-test-slave-8w9v krb5kdc[19785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:48.111930 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:38423
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:38423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:48.437701 19801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:48.438022 19801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:48.438218 19801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:48.445056 19801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:48.445189 19801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:48.445299 19801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:48.445407 19801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:48.445502 19801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:48.455946 19801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:38423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:38423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:48.457297 19801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:48.458736 19801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:48.470476 19807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:48.470582 19806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:48.470603 19809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:48.471410 19801 server_base.cc:1034] running on GCE node
I20260809 15:13:48.472265 19801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:48.473574 19801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:48.474823 19801 hybrid_clock.cc:648] HybridClock initialized: now 1786288428474781 us; error 59 us; skew 500 ppm
I20260809 15:13:48.475219 19801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:48 dist-test-slave-8w9v krb5kdc[19785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288428, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:48.478399 19801 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:48.479596 19801 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:48.479743 19801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:48.480078 19801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:48.482952 19801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "7a4715c32e4e4935a85a458f2fdcf4de"
format_stamp: "Formatted at 2026-08-09 15:13:48 on dist-test-slave-8w9v"
server_key: "87864f1bbd3b144027369c9df3e7180c"
server_key_iv: "121f794b2056178def4c13fc320ab3e4"
server_key_version: "encryptionkey@0"
I20260809 15:13:48.483759 19801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "7a4715c32e4e4935a85a458f2fdcf4de"
format_stamp: "Formatted at 2026-08-09 15:13:48 on dist-test-slave-8w9v"
server_key: "87864f1bbd3b144027369c9df3e7180c"
server_key_iv: "121f794b2056178def4c13fc320ab3e4"
server_key_version: "encryptionkey@0"
I20260809 15:13:48.488978 19801 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:13:48.492588 19817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:48.493891 19801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:13:48.494098 19801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "7a4715c32e4e4935a85a458f2fdcf4de"
format_stamp: "Formatted at 2026-08-09 15:13:48 on dist-test-slave-8w9v"
server_key: "87864f1bbd3b144027369c9df3e7180c"
server_key_iv: "121f794b2056178def4c13fc320ab3e4"
server_key_version: "encryptionkey@0"
I20260809 15:13:48.494379 19801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:48.506086 19801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:48.510828 19801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:48.511126 19801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:48.529486 19801 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:38423
I20260809 15:13:48.529492 19869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:38423 every 8 connection(s)
I20260809 15:13:48.530412 19801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:48.533833 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 19801
I20260809 15:13:48.533928 19870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:48.534088 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:48.534744 15406 external_mini_cluster.cc:1428] Setting key adac653197113e6a0d1cb6b7d9cd3226
I20260809 15:13:48.548007 19870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de: Bootstrap starting.
Aug 09 15:13:48 dist-test-slave-8w9v krb5kdc[19785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:48.551406 19870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:48.552543 19870 log.cc:824] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:48.555528 19870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de: No bootstrap required, opened a new log
I20260809 15:13:48.558869 19873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:48.536798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.15.11.190:38423)
0809 15:13:48.537700 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:48.537716 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:48.537768 (+ 52us) server_negotiation.cc:407] Connection header received
0809 15:13:48.540613 (+ 2845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:48.540641 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:48.541128 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:48.541582 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:48.542926 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:48.544967 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:48.546197 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:48.547133 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:48.547298 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:48.551583 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:48.551593 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:48.551606 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:48.551640 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:48.554693 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:48.555501 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:48.555510 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:48.555516 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:48.555731 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:48.556230 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:48.556234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:48.556236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:48.556749 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:48.556959 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:48.557472 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 15:13:48.557831 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":376,"threads_started":1}
I20260809 15:13:48.564188 19870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a4715c32e4e4935a85a458f2fdcf4de" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38423 } }
I20260809 15:13:48.564548 19870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:48.564667 19870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a4715c32e4e4935a85a458f2fdcf4de, State: Initialized, Role: FOLLOWER
I20260809 15:13:48.565310 19870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a4715c32e4e4935a85a458f2fdcf4de" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38423 } }
I20260809 15:13:48.565552 19870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:48.565722 19870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:48.565902 19870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:48.568249 19870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a4715c32e4e4935a85a458f2fdcf4de" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38423 } }
I20260809 15:13:48.568783 19870 leader_election.cc:302] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a4715c32e4e4935a85a458f2fdcf4de; no voters:
I20260809 15:13:48.569416 19870 leader_election.cc:288] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:48.569574 19875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:48.570248 19875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [term 1 LEADER]: Becoming Leader. State: Replica: 7a4715c32e4e4935a85a458f2fdcf4de, State: Running, Role: LEADER
I20260809 15:13:48.570721 19875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a4715c32e4e4935a85a458f2fdcf4de" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38423 } }
I20260809 15:13:48.571403 19870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:48.573930 19876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a4715c32e4e4935a85a458f2fdcf4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a4715c32e4e4935a85a458f2fdcf4de" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38423 } } }
I20260809 15:13:48.573922 19878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a4715c32e4e4935a85a458f2fdcf4de. Latest consensus state: current_term: 1 leader_uuid: "7a4715c32e4e4935a85a458f2fdcf4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a4715c32e4e4935a85a458f2fdcf4de" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38423 } } }
I20260809 15:13:48.574326 19876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:48.574401 19878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:48.577539 19882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:48.582702 19882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:48.590741 19882 catalog_manager.cc:1347] Generated new cluster ID: 7c67518a27c44c4aa448db4dd435fd4e
I20260809 15:13:48.590848 19882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:48.628374 19882 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:48.629705 19882 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:48.635687 19882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de: Generated new TSK 0
I20260809 15:13:48.636576 19882 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:13:48.701393 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38423
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:49.062017 19898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:49.062412 19898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:49.062623 19898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:49.070227 19898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:49.070335 19898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:49.070453 19898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:49.080730 19898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:49.082021 19898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:49.083328 19898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:49.097188 19904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:49.097143 19903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:49.097101 19906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:49.098187 19898 server_base.cc:1034] running on GCE node
I20260809 15:13:49.098722 19898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:49.099720 19898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:49.100968 19898 hybrid_clock.cc:648] HybridClock initialized: now 1786288429100922 us; error 42 us; skew 500 ppm
I20260809 15:13:49.101296 19898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:49 dist-test-slave-8w9v krb5kdc[19785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288429, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:49.104674 19898 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:13:49.105824 19898 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:49.105921 19898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:49.106267 19898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:49.109551 19898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "633ae73ed4d04d6a8c123a5afa638e14"
format_stamp: "Formatted at 2026-08-09 15:13:49 on dist-test-slave-8w9v"
server_key: "41fb8aaaa2cd948ae87ae0fab485faed"
server_key_iv: "650a04ffb1521ab5004156af742029a6"
server_key_version: "encryptionkey@0"
I20260809 15:13:49.110452 19898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "633ae73ed4d04d6a8c123a5afa638e14"
format_stamp: "Formatted at 2026-08-09 15:13:49 on dist-test-slave-8w9v"
server_key: "41fb8aaaa2cd948ae87ae0fab485faed"
server_key_iv: "650a04ffb1521ab5004156af742029a6"
server_key_version: "encryptionkey@0"
I20260809 15:13:49.115814 19898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 15:13:49.119445 19913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:49.120860 19898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 15:13:49.121006 19898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "633ae73ed4d04d6a8c123a5afa638e14"
format_stamp: "Formatted at 2026-08-09 15:13:49 on dist-test-slave-8w9v"
server_key: "41fb8aaaa2cd948ae87ae0fab485faed"
server_key_iv: "650a04ffb1521ab5004156af742029a6"
server_key_version: "encryptionkey@0"
I20260809 15:13:49.121196 19898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:49.166368 19898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:49.171535 19898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:49.171825 19898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:49.173030 19898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:49.174507 19898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:49.174669 19898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:49.174818 19898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:49.174932 19898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:49.210398 19898 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:40427
I20260809 15:13:49.210448 20032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:40427 every 8 connection(s)
I20260809 15:13:49.211396 19898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:13:49.217568 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 19898
I20260809 15:13:49.217811 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:13:49.218467 15406 external_mini_cluster.cc:1428] Setting key 6bd1a08088e7bea0c250cad09eafd0c7
Aug 09 15:13:49 dist-test-slave-8w9v krb5kdc[19785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288429, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:49.234719 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:49.214033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:37315 (local address 127.15.11.190:38423)
0809 15:13:49.214518 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:49.214523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:49.215560 (+ 1037us) server_negotiation.cc:407] Connection header received
0809 15:13:49.217546 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:49.217554 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:49.217651 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:49.217786 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:49.219581 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:49.220637 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:49.223291 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:49.223677 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:49.223766 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:49.228289 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:49.228298 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:49.228305 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:49.228352 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:49.231219 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:49.232200 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:49.232205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:49.232207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:49.232380 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:49.232896 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:49.232899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:49.232901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:49.233215 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:49.233345 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:49.234279 (+ 934us) server_negotiation.cc:299] Negotiation successful
0809 15:13:49.234450 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":242,"threads_started":1}
I20260809 15:13:49.236342 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:49.214604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38423 (local address 127.15.11.129:37315)
0809 15:13:49.215365 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:49.215421 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:49.217260 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:49.218061 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:49.218070 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:49.218671 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:49.219361 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:49.219388 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:49.220808 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:49.220814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:49.223122 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:49.223142 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:49.223928 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:49.223938 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:49.224327 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:49.225225 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:49.225278 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:49.228012 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:49.231374 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:49.231384 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:49.231409 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:49.231996 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:49.232511 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:49.232519 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:49.232524 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:49.232782 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:49.233380 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:49.233397 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:49.233841 (+ 444us) client_negotiation.cc:769] Sending connection context
0809 15:13:49.234155 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 15:13:49.234780 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":288,"threads_started":1}
I20260809 15:13:49.238035 20033 heartbeater.cc:347] Connected to a master server at 127.15.11.190:38423
I20260809 15:13:49.238380 20033 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:49.239148 20033 heartbeater.cc:511] Master 127.15.11.190:38423 requested a full tablet report, sending...
I20260809 15:13:49.241287 19834 ts_manager.cc:194] Registered new tserver with Master: 633ae73ed4d04d6a8c123a5afa638e14 (127.15.11.129:40427)
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
I20260809 15:13:49.242758 19834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:37315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:13:49.271447 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38423
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:49.599634 20040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:49.599949 20040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:49.600170 20040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:49.607071 20040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:49.607211 20040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:49.607399 20040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:13:49.617121 20040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:49.618361 20040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:49.619802 20040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:49.633198 20045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:49.633213 20048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:49.633189 20046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:49.634208 20040 server_base.cc:1034] running on GCE node
I20260809 15:13:49.634897 20040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:49.635892 20040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:49.637151 20040 hybrid_clock.cc:648] HybridClock initialized: now 1786288429637105 us; error 39 us; skew 500 ppm
I20260809 15:13:49.637492 20040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:49 dist-test-slave-8w9v krb5kdc[19785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288429, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:49.645501 20040 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:13:49.646608 20040 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:49.646696 20040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:49.646958 20040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:49.649868 20040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "07731209060646e187c39b1d96114ec9"
format_stamp: "Formatted at 2026-08-09 15:13:49 on dist-test-slave-8w9v"
server_key: "71b75f9bd04e8ceb6d9cf56c2d86070c"
server_key_iv: "4b26339a13ca8b47a265d1a34a51e997"
server_key_version: "encryptionkey@0"
I20260809 15:13:49.650671 20040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "07731209060646e187c39b1d96114ec9"
format_stamp: "Formatted at 2026-08-09 15:13:49 on dist-test-slave-8w9v"
server_key: "71b75f9bd04e8ceb6d9cf56c2d86070c"
server_key_iv: "4b26339a13ca8b47a265d1a34a51e997"
server_key_version: "encryptionkey@0"
I20260809 15:13:49.655550 20040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 15:13:49.659134 20055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:49.660576 20040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:49.660756 20040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "07731209060646e187c39b1d96114ec9"
format_stamp: "Formatted at 2026-08-09 15:13:49 on dist-test-slave-8w9v"
server_key: "71b75f9bd04e8ceb6d9cf56c2d86070c"
server_key_iv: "4b26339a13ca8b47a265d1a34a51e997"
server_key_version: "encryptionkey@0"
I20260809 15:13:49.660982 20040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:49.678911 20040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:49.682947 20040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:49.683262 20040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:49.684293 20040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:49.685648 20040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:49.685768 20040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:49.685909 20040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:49.686005 20040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:49.721349 20040 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:43613
I20260809 15:13:49.721477 20180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:43613 every 8 connection(s)
I20260809 15:13:49.722270 20040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:13:49.732086 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 20040
I20260809 15:13:49.732326 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:13:49.732846 15406 external_mini_cluster.cc:1428] Setting key 5b9d75b1fa64a6c147b6df4607ac2d26
Aug 09 15:13:49 dist-test-slave-8w9v krb5kdc[19785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288429, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:49.744168 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:49.725037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:43963 (local address 127.15.11.190:38423)
0809 15:13:49.725300 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:49.725307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:49.726643 (+ 1336us) server_negotiation.cc:407] Connection header received
0809 15:13:49.728356 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:49.728361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:49.728438 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:49.728540 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:49.729951 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:49.730708 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:49.732649 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:49.733176 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:49.733311 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:49.737327 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:49.737337 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:49.737346 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:49.737404 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:49.741038 (+ 3634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:49.741777 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:49.741785 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:49.741791 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:49.741949 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:49.742351 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:49.742358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:49.742364 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:49.742745 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:49.742911 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:49.743569 (+ 658us) server_negotiation.cc:299] Negotiation successful
0809 15:13:49.743799 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 15:13:49.745474 20183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:49.725577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38423 (local address 127.15.11.130:43963)
0809 15:13:49.726472 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:49.726515 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:49.728076 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:49.728730 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:49.728738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:49.729217 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:49.729777 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:49.729793 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:49.730854 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:49.730857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:49.732481 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:49.732498 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:49.733435 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:49.733442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:49.733658 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:49.734425 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:49.734447 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:49.737080 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:49.741193 (+ 4113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:49.741199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:49.741210 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:49.741635 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:49.742063 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:49.742066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:49.742069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:49.742228 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:49.742878 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:49.742884 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:49.743173 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 15:13:49.743462 (+ 289us) client_negotiation.cc:240] Negotiation successful
0809 15:13:49.743960 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":444,"threads_started":1}
I20260809 15:13:49.747251 20181 heartbeater.cc:347] Connected to a master server at 127.15.11.190:38423
I20260809 15:13:49.747619 20181 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:49.748540 20181 heartbeater.cc:511] Master 127.15.11.190:38423 requested a full tablet report, sending...
I20260809 15:13:49.750407 19834 ts_manager.cc:194] Registered new tserver with Master: 07731209060646e187c39b1d96114ec9 (127.15.11.130:43613)
I20260809 15:13:49.751681 19834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:43963
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:13:49.790836 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38423
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:50.119652 20186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:50.119910 20186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:50.120014 20186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:50.127135 20186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:50.127236 20186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:50.127347 20186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:13:50.136409 20186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:50.137596 20186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:50.138774 20186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:50.151850 20191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:50.151937 20193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:50.151981 20195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:50.152789 20186 server_base.cc:1034] running on GCE node
I20260809 15:13:50.153321 20186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:50.154158 20186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:50.155367 20186 hybrid_clock.cc:648] HybridClock initialized: now 1786288430155326 us; error 33 us; skew 500 ppm
I20260809 15:13:50.155685 20186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:50 dist-test-slave-8w9v krb5kdc[19785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:50.158882 20186 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:13:50.160092 20186 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:50.160317 20186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:50.160634 20186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:50.163504 20186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "318d48a9c31648478d15fa5b76185562"
format_stamp: "Formatted at 2026-08-09 15:13:50 on dist-test-slave-8w9v"
server_key: "0fc8cf066f5cd83423c37d2a8b7fddca"
server_key_iv: "30fb4d3e8fe5832ce7cffc9bf60a4f60"
server_key_version: "encryptionkey@0"
I20260809 15:13:50.164426 20186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "318d48a9c31648478d15fa5b76185562"
format_stamp: "Formatted at 2026-08-09 15:13:50 on dist-test-slave-8w9v"
server_key: "0fc8cf066f5cd83423c37d2a8b7fddca"
server_key_iv: "30fb4d3e8fe5832ce7cffc9bf60a4f60"
server_key_version: "encryptionkey@0"
I20260809 15:13:50.169376 20186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 15:13:50.172930 20203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:50.174240 20186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:50.174437 20186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "318d48a9c31648478d15fa5b76185562"
format_stamp: "Formatted at 2026-08-09 15:13:50 on dist-test-slave-8w9v"
server_key: "0fc8cf066f5cd83423c37d2a8b7fddca"
server_key_iv: "30fb4d3e8fe5832ce7cffc9bf60a4f60"
server_key_version: "encryptionkey@0"
I20260809 15:13:50.174665 20186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:50.195360 20186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:50.199411 20186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:50.199678 20186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:50.200654 20186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:50.201928 20186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:50.202055 20186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:50.202195 20186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:50.202287 20186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:50.237815 20186 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:38953
I20260809 15:13:50.237910 20327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:38953 every 8 connection(s)
I20260809 15:13:50.238723 20186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:13:50.240965 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 20186
I20260809 15:13:50.241200 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:13:50.241708 15406 external_mini_cluster.cc:1428] Setting key 25e2e52c4576f21e09e95700a155f7e0
I20260809 15:13:50.245644 20033 heartbeater.cc:503] Master 127.15.11.190:38423 was elected leader, sending a full tablet report...
Aug 09 15:13:50 dist-test-slave-8w9v krb5kdc[19785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288430, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:50.259064 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.241329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:58073 (local address 127.15.11.190:38423)
0809 15:13:50.241568 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.241574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.243038 (+ 1464us) server_negotiation.cc:407] Connection header received
0809 15:13:50.245041 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.245046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.245134 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.245266 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:50.246948 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.247778 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.249444 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.249880 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.250032 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.253314 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:50.253320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:50.253324 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:50.253366 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:50.255822 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.256674 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.256680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.256684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.256838 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.257293 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.257298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.257302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.257582 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:50.257760 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.258500 (+ 740us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.258719 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 15:13:50.259996 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.241970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38423 (local address 127.15.11.131:58073)
0809 15:13:50.242868 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:50.242912 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:50.244746 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:50.245510 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:50.245518 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:50.246207 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:50.246754 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.246771 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.247958 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.247962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.249274 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.249287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.250122 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.250132 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.250374 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.250996 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:13:50.251018 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:50.253114 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:50.255972 (+ 2858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:50.255981 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:50.256014 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:50.256533 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:50.256968 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:13:50.256973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:13:50.256977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:13:50.257159 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:13:50.257734 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:50.257742 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:50.258085 (+ 343us) client_negotiation.cc:769] Sending connection context
0809 15:13:50.258373 (+ 288us) client_negotiation.cc:240] Negotiation successful
0809 15:13:50.258891 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":376,"threads_started":1}
I20260809 15:13:50.261579 20328 heartbeater.cc:347] Connected to a master server at 127.15.11.190:38423
I20260809 15:13:50.261931 20328 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:50.262643 20328 heartbeater.cc:511] Master 127.15.11.190:38423 requested a full tablet report, sending...
I20260809 15:13:50.264264 19834 ts_manager.cc:194] Registered new tserver with Master: 318d48a9c31648478d15fa5b76185562 (127.15.11.131:38953)
I20260809 15:13:50.265242 19834 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:58073
I20260809 15:13:50.269574 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:13:50.288249 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.274416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38602 (local address 127.15.11.190:38423)
0809 15:13:50.274642 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.274651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.275141 (+ 490us) server_negotiation.cc:407] Connection header received
0809 15:13:50.275408 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.275413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.275503 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.275623 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:50.277339 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.278385 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.279824 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.280415 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.280568 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.283002 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:50.283010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:50.283015 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:50.283053 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:50.285539 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.286227 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.286232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.286236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.286380 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.286766 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.286771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.286774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.287082 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:50.287375 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.287669 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.287873 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:13:50.292104 19834 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38602:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:13:50.299487 19834 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:50.321473 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.311677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53210 (local address 127.15.11.130:43613)
0809 15:13:50.313202 (+ 1525us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.313212 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.313241 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:13:50.313346 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.313354 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.313688 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.313902 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:50.314652 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.316275 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.319212 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.320218 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.320379 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.320596 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.320957 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.321151 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1344,"thread_start_us":1633,"threads_started":1}
I20260809 15:13:50.321506 20341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.311594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:43613 (local address 127.0.0.1:53210)
0809 15:13:50.312350 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:50.312399 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:50.312568 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:50.313905 (+ 1337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:50.313911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:50.313942 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:50.314433 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.314446 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.316429 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.316434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.319045 (+ 2611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.319057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.320419 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.320426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.320646 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.320713 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 15:13:50.320886 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 15:13:50.321091 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":52,"thread_start_us":364,"threads_started":1}
I20260809 15:13:50.323925 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.313966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:40427 (local address 127.0.0.1:42508)
0809 15:13:50.314566 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:50.314583 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:50.314711 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:50.316646 (+ 1935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:50.316649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:50.316671 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:50.317026 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.317032 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.318730 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.318735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.320982 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.320995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.323158 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.323162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.323268 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.323287 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 15:13:50.323411 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 15:13:50.323532 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":3,"thread_start_us":331,"threads_started":1}
I20260809 15:13:50.323940 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.314465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42508 (local address 127.15.11.129:40427)
0809 15:13:50.315844 (+ 1379us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.315852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.315877 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:13:50.315968 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.315975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.316251 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.316432 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:50.317207 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.318512 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.321151 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.322671 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.322803 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.323037 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.323443 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.323596 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1227,"thread_start_us":1081,"threads_started":1}
I20260809 15:13:50.323971 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.313031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36872 (local address 127.15.11.131:38953)
0809 15:13:50.313985 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.313993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.314024 (+ 31us) server_negotiation.cc:407] Connection header received
0809 15:13:50.314119 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.314127 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.315267 (+ 1140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.316071 (+ 804us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:50.316429 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.318082 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.320925 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.322378 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.323158 (+ 780us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.323386 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.323456 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.323616 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":574,"threads_started":1}
I20260809 15:13:50.324625 20343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.312642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:38953 (local address 127.0.0.1:36872)
0809 15:13:50.313253 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:50.313272 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:50.313391 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:50.315490 (+ 2099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:50.315497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:50.315532 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:50.315887 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.315897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.318231 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.318236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.320717 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.320731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.322623 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.322630 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.322792 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.322822 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:13:50.322888 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 15:13:50.324272 (+ 1384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":113,"thread_start_us":353,"threads_started":1}
I20260809 15:13:50.326122 20109 tablet_service.cc:1467] Processing CreateTablet for tablet b9ccc75b6fa24266a7c22e74da22e84d (DEFAULT_TABLE table=test-table [id=afa0c5ae2550473cbb7638ce91d0bb58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:50.327667 20109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9ccc75b6fa24266a7c22e74da22e84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:50.327947 19966 tablet_service.cc:1467] Processing CreateTablet for tablet b9ccc75b6fa24266a7c22e74da22e84d (DEFAULT_TABLE table=test-table [id=afa0c5ae2550473cbb7638ce91d0bb58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:50.328143 20256 tablet_service.cc:1467] Processing CreateTablet for tablet b9ccc75b6fa24266a7c22e74da22e84d (DEFAULT_TABLE table=test-table [id=afa0c5ae2550473cbb7638ce91d0bb58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:50.329399 19966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9ccc75b6fa24266a7c22e74da22e84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:50.329433 20256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9ccc75b6fa24266a7c22e74da22e84d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:50.339104 20347 tablet_bootstrap.cc:492] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: Bootstrap starting.
I20260809 15:13:50.343116 20347 tablet_bootstrap.cc:654] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:50.343781 20348 tablet_bootstrap.cc:492] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: Bootstrap starting.
I20260809 15:13:50.344813 20349 tablet_bootstrap.cc:492] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: Bootstrap starting.
I20260809 15:13:50.345165 20347 log.cc:824] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:50.347254 20348 tablet_bootstrap.cc:654] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:50.348297 20348 log.cc:824] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:50.349203 20349 tablet_bootstrap.cc:654] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:50.350576 20349 log.cc:824] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:50.351084 20347 tablet_bootstrap.cc:492] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: No bootstrap required, opened a new log
I20260809 15:13:50.351374 20347 ts_tablet_manager.cc:1397] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260809 15:13:50.352231 20348 tablet_bootstrap.cc:492] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: No bootstrap required, opened a new log
I20260809 15:13:50.352514 20348 ts_tablet_manager.cc:1397] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:13:50.353343 20349 tablet_bootstrap.cc:492] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: No bootstrap required, opened a new log
I20260809 15:13:50.353624 20349 ts_tablet_manager.cc:1397] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 15:13:50.360884 20349 raft_consensus.cc:348] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } }
I20260809 15:13:50.361248 20349 raft_consensus.cc:374] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:50.361384 20349 raft_consensus.cc:729] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 318d48a9c31648478d15fa5b76185562, State: Initialized, Role: FOLLOWER
I20260809 15:13:50.362036 20349 consensus_queue.cc:260] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } }
I20260809 15:13:50.362094 20347 raft_consensus.cc:348] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } }
I20260809 15:13:50.362506 20347 raft_consensus.cc:374] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:50.362614 20347 raft_consensus.cc:729] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07731209060646e187c39b1d96114ec9, State: Initialized, Role: FOLLOWER
I20260809 15:13:50.363353 20328 heartbeater.cc:503] Master 127.15.11.190:38423 was elected leader, sending a full tablet report...
I20260809 15:13:50.363190 20347 consensus_queue.cc:260] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } }
I20260809 15:13:50.364194 20349 ts_tablet_manager.cc:1428] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
I20260809 15:13:50.363096 20348 raft_consensus.cc:348] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } }
I20260809 15:13:50.364795 20348 raft_consensus.cc:374] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:50.364921 20348 raft_consensus.cc:729] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 633ae73ed4d04d6a8c123a5afa638e14, State: Initialized, Role: FOLLOWER
I20260809 15:13:50.365554 20181 heartbeater.cc:503] Master 127.15.11.190:38423 was elected leader, sending a full tablet report...
I20260809 15:13:50.365643 20348 consensus_queue.cc:260] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } }
I20260809 15:13:50.367435 20348 ts_tablet_manager.cc:1428] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260809 15:13:50.368898 20347 ts_tablet_manager.cc:1428] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: Time spent starting tablet: real 0.017s user 0.006s sys 0.010s
I20260809 15:13:50.432607 20354 raft_consensus.cc:482] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:50.432871 20354 raft_consensus.cc:504] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } }
I20260809 15:13:50.434530 20354 leader_election.cc:288] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07731209060646e187c39b1d96114ec9 (127.15.11.130:43613), 633ae73ed4d04d6a8c123a5afa638e14 (127.15.11.129:40427)
I20260809 15:13:50.441768 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.434947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:45629 (local address 127.15.11.130:43613)
0809 15:13:50.435154 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.435161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.435457 (+ 296us) server_negotiation.cc:407] Connection header received
0809 15:13:50.435646 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.435651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.435737 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.435828 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:50.436353 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.437408 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.439807 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.440952 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.441147 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.441232 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.441422 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.441553 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:13:50.441763 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.434976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:43613 (local address 127.15.11.131:45629)
0809 15:13:50.435290 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:50.435305 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:50.435489 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:50.435884 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:50.435891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:50.435925 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:50.436213 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.436224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.437555 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.437559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.439655 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.439670 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.441109 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.441113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.441265 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.441288 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:13:50.441359 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 15:13:50.441445 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":52}
I20260809 15:13:50.442256 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.435069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:34351 (local address 127.15.11.129:40427)
0809 15:13:50.435288 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.435296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.435980 (+ 684us) server_negotiation.cc:407] Connection header received
0809 15:13:50.436230 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.436234 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.436327 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.436426 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:13:50.436903 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.437833 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.440102 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.441406 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.441560 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.441621 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.441982 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.442100 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:13:50.442713 20357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.435066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:40427 (local address 127.15.11.131:34351)
0809 15:13:50.435846 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:50.435874 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:50.436016 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:50.436456 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:50.436459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:50.436481 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:13:50.436743 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.436750 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.437990 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.438000 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.439944 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:50.439954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.441609 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.441615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:50.441776 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.441813 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:13:50.442388 (+ 575us) client_negotiation.cc:240] Negotiation successful
0809 15:13:50.442472 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":26,"thread_start_us":459,"threads_started":1}
I20260809 15:13:50.442682 20132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ccc75b6fa24266a7c22e74da22e84d" candidate_uuid: "318d48a9c31648478d15fa5b76185562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07731209060646e187c39b1d96114ec9" is_pre_election: true
I20260809 15:13:50.443244 20132 raft_consensus.cc:2454] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 318d48a9c31648478d15fa5b76185562 in term 0.
I20260809 15:13:50.443806 19986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ccc75b6fa24266a7c22e74da22e84d" candidate_uuid: "318d48a9c31648478d15fa5b76185562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "633ae73ed4d04d6a8c123a5afa638e14" is_pre_election: true
I20260809 15:13:50.444087 20206 leader_election.cc:302] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07731209060646e187c39b1d96114ec9, 318d48a9c31648478d15fa5b76185562; no voters:
I20260809 15:13:50.444454 19986 raft_consensus.cc:2454] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 318d48a9c31648478d15fa5b76185562 in term 0.
I20260809 15:13:50.444525 20354 raft_consensus.cc:2781] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:50.444700 20354 raft_consensus.cc:482] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:50.444815 20354 raft_consensus.cc:3037] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:50.447046 20354 raft_consensus.cc:504] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } }
I20260809 15:13:50.447587 20354 leader_election.cc:288] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [CANDIDATE]: Term 1 election: Requested vote from peers 07731209060646e187c39b1d96114ec9 (127.15.11.130:43613), 633ae73ed4d04d6a8c123a5afa638e14 (127.15.11.129:40427)
I20260809 15:13:50.448063 20132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ccc75b6fa24266a7c22e74da22e84d" candidate_uuid: "318d48a9c31648478d15fa5b76185562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07731209060646e187c39b1d96114ec9"
I20260809 15:13:50.448084 19986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9ccc75b6fa24266a7c22e74da22e84d" candidate_uuid: "318d48a9c31648478d15fa5b76185562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "633ae73ed4d04d6a8c123a5afa638e14"
I20260809 15:13:50.448304 20132 raft_consensus.cc:3037] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:50.448330 19986 raft_consensus.cc:3037] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:50.450435 19986 raft_consensus.cc:2454] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 318d48a9c31648478d15fa5b76185562 in term 1.
I20260809 15:13:50.450459 20132 raft_consensus.cc:2454] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 318d48a9c31648478d15fa5b76185562 in term 1.
I20260809 15:13:50.450822 20206 leader_election.cc:302] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07731209060646e187c39b1d96114ec9, 318d48a9c31648478d15fa5b76185562; no voters:
I20260809 15:13:50.451128 20354 raft_consensus.cc:2781] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:50.451577 20354 raft_consensus.cc:686] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 1 LEADER]: Becoming Leader. State: Replica: 318d48a9c31648478d15fa5b76185562, State: Running, Role: LEADER
I20260809 15:13:50.452052 20354 consensus_queue.cc:237] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } }
I20260809 15:13:50.457166 19832 catalog_manager.cc:5495] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 reported cstate change: term changed from 0 to 1, leader changed from <none> to 318d48a9c31648478d15fa5b76185562 (127.15.11.131). New cstate: current_term: 1 leader_uuid: "318d48a9c31648478d15fa5b76185562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "318d48a9c31648478d15fa5b76185562" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 38953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 } health_report { overall_health: UNKNOWN } } }
W20260809 15:13:50.463809 20034 tablet.cc:2369] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:13:50.474177 20182 tablet.cc:2369] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:50.474581 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.468741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36882 (local address 127.15.11.131:38953)
0809 15:13:50.468955 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.468962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.469194 (+ 232us) server_negotiation.cc:407] Connection header received
0809 15:13:50.469439 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.469443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.469535 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.469682 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:13:50.470044 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.470799 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.472390 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.472845 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.472983 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.473473 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:13:50.473984 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:13:50.474150 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.474232 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.474325 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:13:50.483074 20132 raft_consensus.cc:1261] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [term 1 FOLLOWER]: Refusing update from remote peer 318d48a9c31648478d15fa5b76185562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:50.483073 19986 raft_consensus.cc:1261] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [term 1 FOLLOWER]: Refusing update from remote peer 318d48a9c31648478d15fa5b76185562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:50.484294 20354 consensus_queue.cc:1035] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "633ae73ed4d04d6a8c123a5afa638e14" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:50.484622 20359 consensus_queue.cc:1035] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07731209060646e187c39b1d96114ec9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:13:50.490207 20329 tablet.cc:2369] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:50.509523 19832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:50.509801 19832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38602:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:50.513285 19832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a4715c32e4e4935a85a458f2fdcf4de: Sending DeleteTablet for 3 replicas of tablet b9ccc75b6fa24266a7c22e74da22e84d
I20260809 15:13:50.514380 20109 tablet_service.cc:1514] Processing DeleteTablet for tablet b9ccc75b6fa24266a7c22e74da22e84d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:50 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:53210
I20260809 15:13:50.514487 19966 tablet_service.cc:1514] Processing DeleteTablet for tablet b9ccc75b6fa24266a7c22e74da22e84d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:50 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:42508
I20260809 15:13:50.514467 20256 tablet_service.cc:1514] Processing DeleteTablet for tablet b9ccc75b6fa24266a7c22e74da22e84d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:50 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:36872
I20260809 15:13:50.517486 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 19898
I20260809 15:13:50.528077 20375 tablet_replica.cc:330] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: stopping tablet replica
I20260809 15:13:50.528578 20375 raft_consensus.cc:2229] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:50.528702 20374 tablet_replica.cc:330] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: stopping tablet replica
I20260809 15:13:50.529109 20375 raft_consensus.cc:2258] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:50.529783 20376 tablet_replica.cc:330] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: stopping tablet replica
I20260809 15:13:50.530169 20376 raft_consensus.cc:2229] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:13:50.530817 20376 raft_consensus.cc:2258] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:50.532977 20375 ts_tablet_manager.cc:1905] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:50.535027 20374 raft_consensus.cc:2229] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 09 15:13:50 dist-test-slave-8w9v krb5kdc[19785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:13:50.539042 20374 raft_consensus.cc:2258] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:50.540336 20375 ts_tablet_manager.cc:1918] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:13:50.540512 20375 log.cc:1196] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/b9ccc75b6fa24266a7c22e74da22e84d
I20260809 15:13:50.541107 20375 ts_tablet_manager.cc:1939] T b9ccc75b6fa24266a7c22e74da22e84d P 633ae73ed4d04d6a8c123a5afa638e14: Deleting consensus metadata
I20260809 15:13:50.541468 20374 ts_tablet_manager.cc:1905] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:50.543123 19818 catalog_manager.cc:4841] TS 633ae73ed4d04d6a8c123a5afa638e14 (127.15.11.129:40427): tablet b9ccc75b6fa24266a7c22e74da22e84d (table test-table [id=afa0c5ae2550473cbb7638ce91d0bb58]) successfully deleted
I20260809 15:13:50.543885 20376 ts_tablet_manager.cc:1905] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:50.549149 20374 ts_tablet_manager.cc:1918] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:13:50.549319 20374 log.cc:1196] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/b9ccc75b6fa24266a7c22e74da22e84d
I20260809 15:13:50.549968 20374 ts_tablet_manager.cc:1939] T b9ccc75b6fa24266a7c22e74da22e84d P 07731209060646e187c39b1d96114ec9: Deleting consensus metadata
I20260809 15:13:50.551429 20376 ts_tablet_manager.cc:1918] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:13:50.551604 20376 log.cc:1196] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/b9ccc75b6fa24266a7c22e74da22e84d
I20260809 15:13:50.551636 20346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.518871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.15.11.129:40427)
0809 15:13:50.519114 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.519124 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.523848 (+ 4724us) server_negotiation.cc:407] Connection header received
0809 15:13:50.525108 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.525117 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.525217 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.525341 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:50.530113 (+ 4772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.531205 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.533166 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.533841 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.533930 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.540881 (+ 6951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:50.540895 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:50.540913 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:50.540965 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:50.544912 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.546845 (+ 1933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.546854 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.546858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.547019 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.549457 (+ 2438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.549465 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.549470 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.549982 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:50.550392 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.550981 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.551203 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:13:50.551797 19820 catalog_manager.cc:4841] TS 07731209060646e187c39b1d96114ec9 (127.15.11.130:43613): tablet b9ccc75b6fa24266a7c22e74da22e84d (table test-table [id=afa0c5ae2550473cbb7638ce91d0bb58]) successfully deleted
I20260809 15:13:50.552325 20376 ts_tablet_manager.cc:1939] T b9ccc75b6fa24266a7c22e74da22e84d P 318d48a9c31648478d15fa5b76185562: Deleting consensus metadata
I20260809 15:13:50.553437 20028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:50.554029 19821 catalog_manager.cc:4841] TS 318d48a9c31648478d15fa5b76185562 (127.15.11.131:38953): tablet b9ccc75b6fa24266a7c22e74da22e84d (table test-table [id=afa0c5ae2550473cbb7638ce91d0bb58]) successfully deleted
I20260809 15:13:50.662945 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 19898
I20260809 15:13:50.686700 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 20040
Aug 09 15:13:50 dist-test-slave-8w9v krb5kdc[19785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:13:50.704347 20342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.687571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.15.11.130:43613)
0809 15:13:50.687754 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.687758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.687824 (+ 66us) server_negotiation.cc:407] Connection header received
0809 15:13:50.688031 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.688036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.688102 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.688261 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:50.689824 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.690547 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.692323 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.692711 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.692804 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.697725 (+ 4921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:50.697739 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:50.697755 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:50.697798 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:50.701533 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.702176 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.702186 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.702190 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.702359 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.702738 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.702743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.702747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.703246 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:50.703548 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.703715 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.703957 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:13:50.705142 20175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:50.795126 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 20040
I20260809 15:13:50.824919 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 20186
Aug 09 15:13:50 dist-test-slave-8w9v krb5kdc[19785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:13:50.841655 20344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:50.825816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36886 (local address 127.15.11.131:38953)
0809 15:13:50.826009 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:50.826014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:50.826173 (+ 159us) server_negotiation.cc:407] Connection header received
0809 15:13:50.826399 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:50.826405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:50.826495 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:50.826617 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:50.828067 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.829146 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.830421 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:50.830879 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:50.831043 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:50.835609 (+ 4566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:50.835623 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:50.835641 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:50.835681 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:50.838727 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.839418 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.839426 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.839428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.839591 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:50.839982 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:50.839987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:50.839991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:50.840404 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:50.840760 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:50.841054 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 15:13:50.841299 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:13:50.842399 20323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:50.964648 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 20186
I20260809 15:13:50.989485 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 19801
I20260809 15:13:50.990698 19864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:51.100907 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 19801
2026-08-09T15:13:51Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4336 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T15:13:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:51Z Disabled control of system clock
I20260809 15:13:51.176255 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:46661
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:46661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 15:13:51.503419 20385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:51.503692 20385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:51.503850 20385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:51.510829 20385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:51.510929 20385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:51.510967 20385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:51.511049 20385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:51.511111 20385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:51.520358 20385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:46661
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:46661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:51.521682 20385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:51.523563 20385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:51.535621 20390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:51.535634 20391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:51.535558 20393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:51.536402 20385 server_base.cc:1034] running on GCE node
I20260809 15:13:51.537123 20385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:51.538434 20385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:51.539680 20385 hybrid_clock.cc:648] HybridClock initialized: now 1786288431539644 us; error 56 us; skew 500 ppm
I20260809 15:13:51.540060 20385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:13:51.542413 20385 webserver.cc:460] Webserver started at http://127.15.11.190:36601/ using document root <none> and password file <none>
I20260809 15:13:51.543121 20385 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:51.543267 20385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:51.543610 20385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:51.546599 20385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "2631a72b7fec447787f553c3bc70e350"
format_stamp: "Formatted at 2026-08-09 15:13:51 on dist-test-slave-8w9v"
server_key: "c7772923d7d2217b23334ec09a2486db"
server_key_iv: "0eaab822d6e3c27e8ae3c204510405bc"
server_key_version: "encryptionkey@0"
I20260809 15:13:51.547411 20385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "2631a72b7fec447787f553c3bc70e350"
format_stamp: "Formatted at 2026-08-09 15:13:51 on dist-test-slave-8w9v"
server_key: "c7772923d7d2217b23334ec09a2486db"
server_key_iv: "0eaab822d6e3c27e8ae3c204510405bc"
server_key_version: "encryptionkey@0"
I20260809 15:13:51.552384 20385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 15:13:51.555725 20399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:51.557097 20385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:51.557296 20385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "2631a72b7fec447787f553c3bc70e350"
format_stamp: "Formatted at 2026-08-09 15:13:51 on dist-test-slave-8w9v"
server_key: "c7772923d7d2217b23334ec09a2486db"
server_key_iv: "0eaab822d6e3c27e8ae3c204510405bc"
server_key_version: "encryptionkey@0"
I20260809 15:13:51.557520 20385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:51.563333 20385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:51.564085 20385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:51.564443 20385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:51.582904 20385 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:46661
I20260809 15:13:51.582937 20453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:46661 every 8 connection(s)
I20260809 15:13:51.584299 20385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:51.584781 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 20385
I20260809 15:13:51.585021 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:51.585939 15406 external_mini_cluster.cc:1428] Setting key ed5d0309fdf80b51091964eab00eacf1
I20260809 15:13:51.589058 20454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:51.594260 20458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:51.588080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.15.11.190:46661)
0809 15:13:51.589191 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:51.589202 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:51.589243 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:13:51.590723 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:51.590744 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:51.591127 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:51.591441 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:51.592631 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:51.592637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:51.592649 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:51.592662 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:51.592741 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:51.592760 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:51.592940 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:51.593080 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 15:13:51.593360 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":401,"threads_started":1}
I20260809 15:13:51.600435 20454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350: Bootstrap starting.
I20260809 15:13:51.603363 20454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:51.604450 20454 log.cc:824] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:51.607084 20454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350: No bootstrap required, opened a new log
I20260809 15:13:51.614008 20454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2631a72b7fec447787f553c3bc70e350" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46661 } }
I20260809 15:13:51.614362 20454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:51.614497 20454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2631a72b7fec447787f553c3bc70e350, State: Initialized, Role: FOLLOWER
I20260809 15:13:51.615135 20454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2631a72b7fec447787f553c3bc70e350" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46661 } }
I20260809 15:13:51.615449 20454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:51.615590 20454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:51.615792 20454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:51.618081 20454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2631a72b7fec447787f553c3bc70e350" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46661 } }
I20260809 15:13:51.618577 20454 leader_election.cc:302] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2631a72b7fec447787f553c3bc70e350; no voters:
I20260809 15:13:51.619148 20454 leader_election.cc:288] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:51.619303 20461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:51.620148 20461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [term 1 LEADER]: Becoming Leader. State: Replica: 2631a72b7fec447787f553c3bc70e350, State: Running, Role: LEADER
I20260809 15:13:51.620697 20461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2631a72b7fec447787f553c3bc70e350" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46661 } }
I20260809 15:13:51.621467 20454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:51.623984 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2631a72b7fec447787f553c3bc70e350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2631a72b7fec447787f553c3bc70e350" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46661 } } }
I20260809 15:13:51.624523 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:51.623991 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2631a72b7fec447787f553c3bc70e350. Latest consensus state: current_term: 1 leader_uuid: "2631a72b7fec447787f553c3bc70e350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2631a72b7fec447787f553c3bc70e350" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46661 } } }
I20260809 15:13:51.624871 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:51.626507 20471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:51.630927 20471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:51.638777 20471 catalog_manager.cc:1347] Generated new cluster ID: 35095516ef294191b74364f6b2ebe717
I20260809 15:13:51.638890 20471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:51.653053 20471 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:51.653503 20471 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:51.670616 20471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350: Generated new TSK 0
I20260809 15:13:51.671480 20471 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:13:51.683674 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46661
--builtin_ntp_servers=127.15.11.148:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 15:13:52.042913 20480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:52.043176 20480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:52.043262 20480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:52.050379 20480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:52.050475 20480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:52.050664 20480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:52.059689 20480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:52.061178 20480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:52.063167 20480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:52.076453 20485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:52.076560 20488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:52.076589 20486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:52.077556 20480 server_base.cc:1034] running on GCE node
I20260809 15:13:52.078084 20480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:52.078954 20480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:52.080199 20480 hybrid_clock.cc:648] HybridClock initialized: now 1786288432080123 us; error 53 us; skew 500 ppm
I20260809 15:13:52.080516 20480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:13:52.082849 20480 webserver.cc:460] Webserver started at http://127.15.11.129:40203/ using document root <none> and password file <none>
I20260809 15:13:52.083576 20480 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:52.083696 20480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:52.084057 20480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:52.087005 20480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "0b2e397d66994d3ab942daaa74167db9"
format_stamp: "Formatted at 2026-08-09 15:13:52 on dist-test-slave-8w9v"
server_key: "89e98deca2b8da3ff27d665f52502387"
server_key_iv: "0949558eb43f5a11ec95714a055c479f"
server_key_version: "encryptionkey@0"
I20260809 15:13:52.087827 20480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "0b2e397d66994d3ab942daaa74167db9"
format_stamp: "Formatted at 2026-08-09 15:13:52 on dist-test-slave-8w9v"
server_key: "89e98deca2b8da3ff27d665f52502387"
server_key_iv: "0949558eb43f5a11ec95714a055c479f"
server_key_version: "encryptionkey@0"
I20260809 15:13:52.092991 20480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 15:13:52.096719 20494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:52.098050 20480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:52.098230 20480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "0b2e397d66994d3ab942daaa74167db9"
format_stamp: "Formatted at 2026-08-09 15:13:52 on dist-test-slave-8w9v"
server_key: "89e98deca2b8da3ff27d665f52502387"
server_key_iv: "0949558eb43f5a11ec95714a055c479f"
server_key_version: "encryptionkey@0"
I20260809 15:13:52.098465 20480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:52.104382 20480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:52.105181 20480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:52.105463 20480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:52.106405 20480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:52.107841 20480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:52.107964 20480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:52.108096 20480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:52.108213 20480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 15:13:52.142603 20480 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:43357
I20260809 15:13:52.142656 20614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:43357 every 8 connection(s)
I20260809 15:13:52.143993 20480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:13:52.146899 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 20480
I20260809 15:13:52.147090 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:13:52.147974 15406 external_mini_cluster.cc:1428] Setting key a3c3a7c68892f015d8574c75787a09ad
I20260809 15:13:52.151172 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46661
--builtin_ntp_servers=127.15.11.148:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 15:13:52.153177 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:52.146575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:45733 (local address 127.15.11.190:46661)
0809 15:13:52.147108 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:52.147112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:52.148572 (+ 1460us) server_negotiation.cc:407] Connection header received
0809 15:13:52.150782 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:52.150786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:52.150834 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:52.150920 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:52.152463 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:52.152468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:52.152471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:52.152487 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:52.152532 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:52.152543 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:52.152641 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:52.152885 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 15:13:52.152934 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":268,"threads_started":1}
I20260809 15:13:52.154529 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:52.147321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:46661 (local address 127.15.11.129:45733)
0809 15:13:52.148398 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:52.148443 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:52.150489 (+ 2046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:52.151206 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:52.151219 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:52.151351 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:52.152043 (+ 692us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:52.152049 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:52.152104 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:52.152267 (+ 163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:52.152654 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:52.152668 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:52.152679 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 15:13:52.152973 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 15:13:52.153243 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":320,"threads_started":1}
I20260809 15:13:52.156069 20615 heartbeater.cc:347] Connected to a master server at 127.15.11.190:46661
I20260809 15:13:52.156412 20615 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:52.157023 20615 heartbeater.cc:511] Master 127.15.11.190:46661 requested a full tablet report, sending...
I20260809 15:13:52.159221 20418 ts_manager.cc:194] Registered new tserver with Master: 0b2e397d66994d3ab942daaa74167db9 (127.15.11.129:43357)
W20260809 15:13:52.484812 20620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:52.485074 20620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:52.485159 20620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:52.492022 20620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:52.492172 20620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:52.492314 20620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:13:52.501045 20620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:52.502388 20620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:52.504114 20620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:52.517488 20628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:52.517444 20625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:52.518050 20626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:52.526051 20620 server_base.cc:1034] running on GCE node
I20260809 15:13:52.526643 20620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:52.527601 20620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:52.528839 20620 hybrid_clock.cc:648] HybridClock initialized: now 1786288432528803 us; error 53 us; skew 500 ppm
I20260809 15:13:52.529162 20620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:13:52.531735 20620 webserver.cc:460] Webserver started at http://127.15.11.130:41611/ using document root <none> and password file <none>
I20260809 15:13:52.532582 20620 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:52.532719 20620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:52.533093 20620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:52.535866 20620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "855dcf059ea847f3a8ff5ed15c52e113"
format_stamp: "Formatted at 2026-08-09 15:13:52 on dist-test-slave-8w9v"
server_key: "a9298d5ba377dc2937419e742c5297f6"
server_key_iv: "05cf2f9139472fbb32d7cc20464b7bbc"
server_key_version: "encryptionkey@0"
I20260809 15:13:52.536907 20620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "855dcf059ea847f3a8ff5ed15c52e113"
format_stamp: "Formatted at 2026-08-09 15:13:52 on dist-test-slave-8w9v"
server_key: "a9298d5ba377dc2937419e742c5297f6"
server_key_iv: "05cf2f9139472fbb32d7cc20464b7bbc"
server_key_version: "encryptionkey@0"
I20260809 15:13:52.542536 20620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:13:52.545909 20634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:52.547132 20620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:13:52.547302 20620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "855dcf059ea847f3a8ff5ed15c52e113"
format_stamp: "Formatted at 2026-08-09 15:13:52 on dist-test-slave-8w9v"
server_key: "a9298d5ba377dc2937419e742c5297f6"
server_key_iv: "05cf2f9139472fbb32d7cc20464b7bbc"
server_key_version: "encryptionkey@0"
I20260809 15:13:52.547546 20620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:52.552928 20620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:52.553714 20620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:52.553984 20620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:52.554867 20620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:52.556380 20620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:52.556469 20620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:52.556623 20620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:52.556716 20620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:52.590480 20620 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:39259
I20260809 15:13:52.590572 20757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:39259 every 8 connection(s)
I20260809 15:13:52.591756 20620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:13:52.595978 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 20620
I20260809 15:13:52.596210 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:13:52.596733 15406 external_mini_cluster.cc:1428] Setting key 8303a771895df6031d6bb45e0678bddc
I20260809 15:13:52.599622 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46661
--builtin_ntp_servers=127.15.11.148:45157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 15:13:52.601423 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:52.594657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:53365 (local address 127.15.11.190:46661)
0809 15:13:52.594868 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:52.594874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:52.596455 (+ 1581us) server_negotiation.cc:407] Connection header received
0809 15:13:52.598836 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:52.598841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:52.598902 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:52.599019 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:52.600548 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:52.600557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:52.600565 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:52.600588 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:52.600633 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:52.600650 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:52.600774 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:52.601012 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 15:13:52.601078 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:13:52.602384 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:52.595117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:46661 (local address 127.15.11.130:53365)
0809 15:13:52.596281 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:52.596332 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:52.598546 (+ 2214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:52.599269 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:52.599279 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:52.599396 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:52.600201 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:52.600205 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:52.600249 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:52.600349 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:52.600769 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:52.600779 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:52.600786 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 15:13:52.601054 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 15:13:52.601279 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":333,"threads_started":1}
I20260809 15:13:52.603806 20758 heartbeater.cc:347] Connected to a master server at 127.15.11.190:46661
I20260809 15:13:52.604110 20758 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:52.604730 20758 heartbeater.cc:511] Master 127.15.11.190:46661 requested a full tablet report, sending...
I20260809 15:13:52.606071 20418 ts_manager.cc:194] Registered new tserver with Master: 855dcf059ea847f3a8ff5ed15c52e113 (127.15.11.130:39259)
W20260809 15:13:52.931881 20761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:52.932219 20761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:52.932379 20761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:52.939522 20761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:52.939612 20761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:52.939827 20761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:13:52.948899 20761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:45157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:46661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:52.950191 20761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:52.952270 20761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:52.965761 20766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:52.965731 20767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:52.965947 20769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:52.966576 20761 server_base.cc:1034] running on GCE node
I20260809 15:13:52.967193 20761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:52.968060 20761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:52.969343 20761 hybrid_clock.cc:648] HybridClock initialized: now 1786288432969283 us; error 51 us; skew 500 ppm
I20260809 15:13:52.969666 20761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:13:52.971910 20761 webserver.cc:460] Webserver started at http://127.15.11.131:45111/ using document root <none> and password file <none>
I20260809 15:13:52.972678 20761 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:52.972810 20761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:52.973153 20761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:52.975940 20761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "66a5f5883e3f4cb6b2155239864c7269"
format_stamp: "Formatted at 2026-08-09 15:13:52 on dist-test-slave-8w9v"
server_key: "207fbdaaa51a772a84740b2e68d64d33"
server_key_iv: "04a7f6cabe5cf04ec2ceac025f8f0adc"
server_key_version: "encryptionkey@0"
I20260809 15:13:52.976835 20761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "66a5f5883e3f4cb6b2155239864c7269"
format_stamp: "Formatted at 2026-08-09 15:13:52 on dist-test-slave-8w9v"
server_key: "207fbdaaa51a772a84740b2e68d64d33"
server_key_iv: "04a7f6cabe5cf04ec2ceac025f8f0adc"
server_key_version: "encryptionkey@0"
I20260809 15:13:52.981853 20761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:13:52.985204 20776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:52.986503 20761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:52.986704 20761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "66a5f5883e3f4cb6b2155239864c7269"
format_stamp: "Formatted at 2026-08-09 15:13:52 on dist-test-slave-8w9v"
server_key: "207fbdaaa51a772a84740b2e68d64d33"
server_key_iv: "04a7f6cabe5cf04ec2ceac025f8f0adc"
server_key_version: "encryptionkey@0"
I20260809 15:13:52.986953 20761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:52.992488 20761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:52.993273 20761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:52.993575 20761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:52.994716 20761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:13:52.996315 20761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:13:52.996443 20761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:52.996580 20761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:13:52.996680 20761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:53.030087 20761 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:41929
I20260809 15:13:53.030139 20898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:41929 every 8 connection(s)
I20260809 15:13:53.031486 20761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:13:53.033373 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 20761
I20260809 15:13:53.033543 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:13:53.034135 15406 external_mini_cluster.cc:1428] Setting key 0a5597808f305d00ae5e210442fc6719
I20260809 15:13:53.041021 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.034733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:56129 (local address 127.15.11.190:46661)
0809 15:13:53.034959 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.034966 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.036769 (+ 1803us) server_negotiation.cc:407] Connection header received
0809 15:13:53.038691 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.038696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.038763 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.038883 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.040227 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.040233 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.040238 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.040256 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.040300 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.040312 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.040439 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.040644 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.040715 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:13:53.041884 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.035380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:46661 (local address 127.15.11.131:56129)
0809 15:13:53.036616 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:53.036652 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:53.038430 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:53.039110 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:53.039118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:53.039203 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:53.039914 (+ 711us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:53.039916 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:53.039953 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:53.040037 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:53.040417 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:53.040428 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:53.040435 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 15:13:53.040729 (+ 294us) client_negotiation.cc:240] Negotiation successful
0809 15:13:53.040974 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":346,"threads_started":1}
I20260809 15:13:53.043193 20901 heartbeater.cc:347] Connected to a master server at 127.15.11.190:46661
I20260809 15:13:53.043452 20901 heartbeater.cc:464] Registering TS with master...
I20260809 15:13:53.043948 20901 heartbeater.cc:511] Master 127.15.11.190:46661 requested a full tablet report, sending...
I20260809 15:13:53.045323 20418 ts_manager.cc:194] Registered new tserver with Master: 66a5f5883e3f4cb6b2155239864c7269 (127.15.11.131:41929)
I20260809 15:13:53.049583 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:13:53.056981 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.054336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.15.11.190:46661)
0809 15:13:53.054549 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.054558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.055030 (+ 472us) server_negotiation.cc:407] Connection header received
0809 15:13:53.055151 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.055155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.055202 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.055278 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.056208 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.056212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.056215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.056229 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.056262 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.056272 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.056430 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.056571 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.056633 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:13:53.061309 20418 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50786:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:13:53.068003 20418 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:13:53.095441 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.081280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.15.11.130:39259)
0809 15:13:53.090469 (+ 9189us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.090480 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.090517 (+ 37us) server_negotiation.cc:407] Connection header received
0809 15:13:53.091008 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.091019 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.091265 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.091417 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.093673 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.093685 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.093701 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.093731 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.093791 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.093822 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.094003 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.094968 (+ 965us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.095076 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9027,"thread_start_us":8922,"threads_started":1}
I20260809 15:13:53.095520 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.080066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.15.11.131:41929)
0809 15:13:53.090860 (+ 10794us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.090869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.090903 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:13:53.091057 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.091063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.091256 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.091439 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.093987 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.093995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.094007 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.094027 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.094091 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.094110 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.094267 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.095042 (+ 775us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.095119 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10638,"thread_start_us":10483,"threads_started":1}
I20260809 15:13:53.095801 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.080529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:39259 (local address 127.0.0.1:39400)
0809 15:13:53.089935 (+ 9406us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:53.089957 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:53.090802 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:53.091972 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:53.091982 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:53.092098 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:53.093333 (+ 1235us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:53.093337 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:53.093393 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:53.093478 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:53.094057 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:53.094069 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:53.094077 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:13:53.094259 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 15:13:53.094358 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9248,"thread_start_us":9080,"threads_started":1}
I20260809 15:13:53.095947 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.079831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:41929 (local address 127.0.0.1:53628)
0809 15:13:53.080590 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:53.080630 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:53.080765 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:53.092886 (+ 12121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:53.092890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:53.092912 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:53.093757 (+ 845us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:53.093760 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:53.093776 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:53.093827 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:53.094319 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:53.094323 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:53.094327 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:13:53.094497 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 15:13:53.094563 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":343,"threads_started":1}
I20260809 15:13:53.097570 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.080977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:43357 (local address 127.0.0.1:34024)
0809 15:13:53.094582 (+ 13605us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:53.094602 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:53.094709 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:53.095442 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:53.095446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:53.095468 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:53.096382 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:53.096387 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:53.096403 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:53.096445 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:53.096845 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:53.096850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:53.096854 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:13:53.097004 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 15:13:53.097057 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13469,"thread_start_us":13330,"threads_started":1}
I20260809 15:13:53.098410 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.081337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.15.11.129:43357)
0809 15:13:53.093942 (+ 12605us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.093953 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.094853 (+ 900us) server_negotiation.cc:407] Connection header received
0809 15:13:53.094942 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.094950 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.095163 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.095406 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.096622 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.096630 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.096647 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.096670 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.096713 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.096727 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.096883 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.096990 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.098097 (+ 1107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12356,"thread_start_us":12609,"threads_started":1}
I20260809 15:13:53.100035 20687 tablet_service.cc:1467] Processing CreateTablet for tablet ed40131c6805482980bf9e556cef0ce4 (DEFAULT_TABLE table=test-table [id=4bd24bcbecde486da28bc4c786f42124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:53.100152 20828 tablet_service.cc:1467] Processing CreateTablet for tablet ed40131c6805482980bf9e556cef0ce4 (DEFAULT_TABLE table=test-table [id=4bd24bcbecde486da28bc4c786f42124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:53.101548 20828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed40131c6805482980bf9e556cef0ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:53.101557 20687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed40131c6805482980bf9e556cef0ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:53.101799 20545 tablet_service.cc:1467] Processing CreateTablet for tablet ed40131c6805482980bf9e556cef0ce4 (DEFAULT_TABLE table=test-table [id=4bd24bcbecde486da28bc4c786f42124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:13:53.103101 20545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ed40131c6805482980bf9e556cef0ce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:53.112973 20920 tablet_bootstrap.cc:492] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: Bootstrap starting.
I20260809 15:13:53.115953 20921 tablet_bootstrap.cc:492] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: Bootstrap starting.
I20260809 15:13:53.116523 20920 tablet_bootstrap.cc:654] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:53.118108 20920 log.cc:824] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:53.119603 20921 tablet_bootstrap.cc:654] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:53.120345 20922 tablet_bootstrap.cc:492] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: Bootstrap starting.
I20260809 15:13:53.120813 20921 log.cc:824] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:53.121492 20920 tablet_bootstrap.cc:492] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: No bootstrap required, opened a new log
I20260809 15:13:53.121789 20920 ts_tablet_manager.cc:1397] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 15:13:53.123750 20921 tablet_bootstrap.cc:492] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: No bootstrap required, opened a new log
I20260809 15:13:53.123901 20922 tablet_bootstrap.cc:654] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:53.124068 20921 ts_tablet_manager.cc:1397] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 15:13:53.125093 20922 log.cc:824] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:53.131549 20921 raft_consensus.cc:348] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.131990 20921 raft_consensus.cc:374] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:53.132110 20921 raft_consensus.cc:729] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2e397d66994d3ab942daaa74167db9, State: Initialized, Role: FOLLOWER
I20260809 15:13:53.132818 20921 consensus_queue.cc:260] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.133713 20922 tablet_bootstrap.cc:492] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: No bootstrap required, opened a new log
I20260809 15:13:53.133420 20920 raft_consensus.cc:348] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.133842 20920 raft_consensus.cc:374] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:53.133937 20920 raft_consensus.cc:729] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855dcf059ea847f3a8ff5ed15c52e113, State: Initialized, Role: FOLLOWER
I20260809 15:13:53.133987 20922 ts_tablet_manager.cc:1397] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260809 15:13:53.134011 20615 heartbeater.cc:503] Master 127.15.11.190:46661 was elected leader, sending a full tablet report...
I20260809 15:13:53.134593 20921 ts_tablet_manager.cc:1428] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: Time spent starting tablet: real 0.010s user 0.003s sys 0.007s
I20260809 15:13:53.134608 20920 consensus_queue.cc:260] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.135646 20758 heartbeater.cc:503] Master 127.15.11.190:46661 was elected leader, sending a full tablet report...
I20260809 15:13:53.136307 20920 ts_tablet_manager.cc:1428] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260809 15:13:53.143682 20922 raft_consensus.cc:348] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.144130 20922 raft_consensus.cc:374] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:53.144251 20922 raft_consensus.cc:729] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66a5f5883e3f4cb6b2155239864c7269, State: Initialized, Role: FOLLOWER
I20260809 15:13:53.144855 20922 consensus_queue.cc:260] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.145897 20928 raft_consensus.cc:482] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:53.145984 20901 heartbeater.cc:503] Master 127.15.11.190:46661 was elected leader, sending a full tablet report...
I20260809 15:13:53.146242 20928 raft_consensus.cc:504] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.146539 20922 ts_tablet_manager.cc:1428] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: Time spent starting tablet: real 0.012s user 0.011s sys 0.003s
W20260809 15:13:53.146677 20617 tablet.cc:2369] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:13:53.148351 20926 raft_consensus.cc:482] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:13:53.148638 20926 raft_consensus.cc:504] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.149159 20928 leader_election.cc:288] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b2e397d66994d3ab942daaa74167db9 (127.15.11.129:43357), 855dcf059ea847f3a8ff5ed15c52e113 (127.15.11.130:39259)
I20260809 15:13:53.150466 20926 leader_election.cc:288] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66a5f5883e3f4cb6b2155239864c7269 (127.15.11.131:41929), 855dcf059ea847f3a8ff5ed15c52e113 (127.15.11.130:39259)
I20260809 15:13:53.153190 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.148487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:43357 (local address 127.15.11.131:58489)
0809 15:13:53.148853 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:53.148872 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:53.148956 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:53.150266 (+ 1310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:53.150270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:53.150322 (+ 52us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:53.151903 (+ 1581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:53.151909 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:53.151929 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:53.151981 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:53.152683 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:53.152690 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:53.152695 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:13:53.152808 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 15:13:53.152871 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239}
I20260809 15:13:53.155031 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.149178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:58489 (local address 127.15.11.129:43357)
0809 15:13:53.149476 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.149482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.149500 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:13:53.149550 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.149554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.149600 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.149684 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.152221 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.152226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.152230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.152247 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.152285 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.152304 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.152424 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.154578 (+ 2154us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.154669 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260809 15:13:53.157335 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.149603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:39259 (local address 127.15.11.131:44275)
0809 15:13:53.153487 (+ 3884us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:53.153507 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:53.153598 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:53.155453 (+ 1855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:53.155461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:53.155493 (+ 32us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:53.156208 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:53.156214 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:53.156236 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:53.156292 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:53.156773 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:53.156780 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:53.156788 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:13:53.156890 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 15:13:53.156949 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3718,"threads_started":1}
I20260809 15:13:53.157998 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.150114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:44275 (local address 127.15.11.130:39259)
0809 15:13:53.150988 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.150993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.154018 (+ 3025us) server_negotiation.cc:407] Connection header received
0809 15:13:53.154088 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.154096 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.154163 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.154260 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.156490 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.156498 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.156506 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.156531 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.156577 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.156597 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.156722 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.157618 (+ 896us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.157688 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712}
I20260809 15:13:53.166158 20566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed40131c6805482980bf9e556cef0ce4" candidate_uuid: "66a5f5883e3f4cb6b2155239864c7269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2e397d66994d3ab942daaa74167db9" is_pre_election: true
I20260809 15:13:53.166675 20566 raft_consensus.cc:2454] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a5f5883e3f4cb6b2155239864c7269 in term 0.
I20260809 15:13:53.167418 20777 leader_election.cc:302] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2e397d66994d3ab942daaa74167db9, 66a5f5883e3f4cb6b2155239864c7269; no voters:
I20260809 15:13:53.167898 20928 raft_consensus.cc:2781] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:53.168143 20928 raft_consensus.cc:482] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:53.167912 20708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed40131c6805482980bf9e556cef0ce4" candidate_uuid: "66a5f5883e3f4cb6b2155239864c7269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855dcf059ea847f3a8ff5ed15c52e113" is_pre_election: true
I20260809 15:13:53.168323 20928 raft_consensus.cc:3037] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:53.168356 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.152641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:46417 (local address 127.15.11.130:39259)
0809 15:13:53.158261 (+ 5620us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.158268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.165770 (+ 7502us) server_negotiation.cc:407] Connection header received
0809 15:13:53.165957 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.165961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.166032 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.166131 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.167465 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.167470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.167474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.167514 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.167551 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.167566 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.167678 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.167889 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.167957 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5497,"thread_start_us":13743,"threads_started":1}
I20260809 15:13:53.168463 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.152213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:39259 (local address 127.15.11.129:46417)
0809 15:13:53.165667 (+ 13454us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:53.165689 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:53.165819 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:53.166412 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:53.166418 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:53.166445 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:53.167212 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:53.167215 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:53.167231 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:53.167291 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:53.167777 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:53.167782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:53.167787 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:13:53.167929 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 15:13:53.167979 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13296,"thread_start_us":13065,"threads_started":1}
I20260809 15:13:53.168570 20708 raft_consensus.cc:2454] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66a5f5883e3f4cb6b2155239864c7269 in term 0.
I20260809 15:13:53.169111 20707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed40131c6805482980bf9e556cef0ce4" candidate_uuid: "0b2e397d66994d3ab942daaa74167db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855dcf059ea847f3a8ff5ed15c52e113" is_pre_election: true
I20260809 15:13:53.169420 20707 raft_consensus.cc:2454] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b2e397d66994d3ab942daaa74167db9 in term 0.
I20260809 15:13:53.170025 20497 leader_election.cc:302] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b2e397d66994d3ab942daaa74167db9, 855dcf059ea847f3a8ff5ed15c52e113; no voters:
I20260809 15:13:53.170426 20926 raft_consensus.cc:2781] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:13:53.170588 20926 raft_consensus.cc:482] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:13:53.170701 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.166448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:46527 (local address 127.15.11.131:41929)
0809 15:13:53.166608 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.166615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.166633 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:13:53.166683 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.166689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.166742 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.166849 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.170006 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.170018 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.170023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.170043 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.170083 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.170099 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.170242 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.170399 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.170455 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:13:53.170720 20926 raft_consensus.cc:3037] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:53.171430 20930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.151443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:41929 (local address 127.15.11.129:46527)
0809 15:13:53.164847 (+ 13404us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:53.164869 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:53.165014 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:53.167197 (+ 2183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:53.167201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:53.167225 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:13:53.169777 (+ 2552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:13:53.169781 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:13:53.169798 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:13:53.169842 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:13:53.170228 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:13:53.170233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:13:53.170239 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:13:53.170954 (+ 715us) client_negotiation.cc:240] Negotiation successful
0809 15:13:53.171007 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13239,"thread_start_us":12990,"threads_started":1}
I20260809 15:13:53.171782 20928 raft_consensus.cc:504] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.172473 20928 leader_election.cc:288] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [CANDIDATE]: Term 1 election: Requested vote from peers 0b2e397d66994d3ab942daaa74167db9 (127.15.11.129:43357), 855dcf059ea847f3a8ff5ed15c52e113 (127.15.11.130:39259)
I20260809 15:13:53.172864 20566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed40131c6805482980bf9e556cef0ce4" candidate_uuid: "66a5f5883e3f4cb6b2155239864c7269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b2e397d66994d3ab942daaa74167db9"
I20260809 15:13:53.172925 20707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed40131c6805482980bf9e556cef0ce4" candidate_uuid: "66a5f5883e3f4cb6b2155239864c7269" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855dcf059ea847f3a8ff5ed15c52e113"
I20260809 15:13:53.173091 20707 raft_consensus.cc:3037] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:53.173470 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed40131c6805482980bf9e556cef0ce4" candidate_uuid: "0b2e397d66994d3ab942daaa74167db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a5f5883e3f4cb6b2155239864c7269" is_pre_election: true
I20260809 15:13:53.173915 20849 raft_consensus.cc:2379] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0b2e397d66994d3ab942daaa74167db9 in current term 1: Already voted for candidate 66a5f5883e3f4cb6b2155239864c7269 in this term.
I20260809 15:13:53.174379 20926 raft_consensus.cc:504] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.174815 20566 raft_consensus.cc:2379] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 66a5f5883e3f4cb6b2155239864c7269 in current term 1: Already voted for candidate 0b2e397d66994d3ab942daaa74167db9 in this term.
I20260809 15:13:53.175155 20926 leader_election.cc:288] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [CANDIDATE]: Term 1 election: Requested vote from peers 66a5f5883e3f4cb6b2155239864c7269 (127.15.11.131:41929), 855dcf059ea847f3a8ff5ed15c52e113 (127.15.11.130:39259)
I20260809 15:13:53.175565 20849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed40131c6805482980bf9e556cef0ce4" candidate_uuid: "0b2e397d66994d3ab942daaa74167db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66a5f5883e3f4cb6b2155239864c7269"
I20260809 15:13:53.175726 20708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ed40131c6805482980bf9e556cef0ce4" candidate_uuid: "0b2e397d66994d3ab942daaa74167db9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "855dcf059ea847f3a8ff5ed15c52e113"
I20260809 15:13:53.175814 20849 raft_consensus.cc:2379] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0b2e397d66994d3ab942daaa74167db9 in current term 1: Already voted for candidate 66a5f5883e3f4cb6b2155239864c7269 in this term.
I20260809 15:13:53.176419 20707 raft_consensus.cc:2454] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66a5f5883e3f4cb6b2155239864c7269 in term 1.
I20260809 15:13:53.176635 20497 leader_election.cc:302] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b2e397d66994d3ab942daaa74167db9; no voters: 66a5f5883e3f4cb6b2155239864c7269, 855dcf059ea847f3a8ff5ed15c52e113
I20260809 15:13:53.176790 20779 leader_election.cc:302] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 66a5f5883e3f4cb6b2155239864c7269, 855dcf059ea847f3a8ff5ed15c52e113; no voters: 0b2e397d66994d3ab942daaa74167db9
I20260809 15:13:53.176983 20926 raft_consensus.cc:2726] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 15:13:53.177115 20928 raft_consensus.cc:2781] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:53.177601 20928 raft_consensus.cc:686] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 1 LEADER]: Becoming Leader. State: Replica: 66a5f5883e3f4cb6b2155239864c7269, State: Running, Role: LEADER
I20260809 15:13:53.178069 20928 consensus_queue.cc:237] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } }
I20260809 15:13:53.182686 20418 catalog_manager.cc:5495] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66a5f5883e3f4cb6b2155239864c7269 (127.15.11.131). New cstate: current_term: 1 leader_uuid: "66a5f5883e3f4cb6b2155239864c7269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66a5f5883e3f4cb6b2155239864c7269" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 41929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 } health_report { overall_health: UNKNOWN } } }
I20260809 15:13:53.209759 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.207434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.15.11.131:41929)
0809 15:13:53.207628 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.207634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.207750 (+ 116us) server_negotiation.cc:407] Connection header received
0809 15:13:53.208010 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.208013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.208116 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.208276 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.209115 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.209120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.209126 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.209147 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.209177 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.209188 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.209324 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.209421 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.209481 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:13:53.217795 20566 raft_consensus.cc:1261] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 1 FOLLOWER]: Refusing update from remote peer 66a5f5883e3f4cb6b2155239864c7269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:53.217809 20707 raft_consensus.cc:1261] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 1 FOLLOWER]: Refusing update from remote peer 66a5f5883e3f4cb6b2155239864c7269: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:13:53.218611 20933 consensus_queue.cc:1035] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b2e397d66994d3ab942daaa74167db9" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 43357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:53.218972 20928 consensus_queue.cc:1035] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "855dcf059ea847f3a8ff5ed15c52e113" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:13:53.247977 20417 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:53.248345 20417 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50786:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:13:53.252226 20417 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2631a72b7fec447787f553c3bc70e350: Sending DeleteTablet for 3 replicas of tablet ed40131c6805482980bf9e556cef0ce4
I20260809 15:13:53.253453 20828 tablet_service.cc:1514] Processing DeleteTablet for tablet ed40131c6805482980bf9e556cef0ce4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:53 UTC) from {username='slave'} at 127.0.0.1:53628
I20260809 15:13:53.253597 20545 tablet_service.cc:1514] Processing DeleteTablet for tablet ed40131c6805482980bf9e556cef0ce4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:53 UTC) from {username='slave'} at 127.0.0.1:34024
I20260809 15:13:53.254410 20949 tablet_replica.cc:330] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: stopping tablet replica
I20260809 15:13:53.254624 20687 tablet_service.cc:1514] Processing DeleteTablet for tablet ed40131c6805482980bf9e556cef0ce4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:13:53 UTC) from {username='slave'} at 127.0.0.1:39400
I20260809 15:13:53.254878 20949 raft_consensus.cc:2229] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:53.255194 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 20480
I20260809 15:13:53.256275 20949 raft_consensus.cc:2258] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:53.258562 20949 ts_tablet_manager.cc:1905] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:53.265919 20949 ts_tablet_manager.cc:1918] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:13:53.266165 20949 log.cc:1196] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/ed40131c6805482980bf9e556cef0ce4
I20260809 15:13:53.266801 20949 ts_tablet_manager.cc:1939] T ed40131c6805482980bf9e556cef0ce4 P 0b2e397d66994d3ab942daaa74167db9: Deleting consensus metadata
I20260809 15:13:53.268834 20401 catalog_manager.cc:4841] TS 0b2e397d66994d3ab942daaa74167db9 (127.15.11.129:43357): tablet ed40131c6805482980bf9e556cef0ce4 (table test-table [id=4bd24bcbecde486da28bc4c786f42124]) successfully deleted
I20260809 15:13:53.269760 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.256640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.15.11.129:43357)
0809 15:13:53.256866 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.256875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.266766 (+ 9891us) server_negotiation.cc:407] Connection header received
0809 15:13:53.267274 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.267283 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.267448 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.267535 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.268923 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.268945 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.268953 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.268976 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.269020 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.269039 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.269145 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.269410 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.269474 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 15:13:53.270597 20610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:53.276744 20950 tablet_replica.cc:330] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: stopping tablet replica
I20260809 15:13:53.277357 20950 raft_consensus.cc:2229] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:13:53.277995 20950 raft_consensus.cc:2258] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:53.278666 20948 tablet_replica.cc:330] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: stopping tablet replica
I20260809 15:13:53.279134 20948 raft_consensus.cc:2229] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:13:53.279820 20948 raft_consensus.cc:2258] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:13:53.282104 20948 ts_tablet_manager.cc:1905] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:53.282219 20950 ts_tablet_manager.cc:1905] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:13:53.289491 20948 ts_tablet_manager.cc:1918] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:13:53.289722 20948 log.cc:1196] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/ed40131c6805482980bf9e556cef0ce4
I20260809 15:13:53.290094 20950 ts_tablet_manager.cc:1918] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:13:53.290297 20950 log.cc:1196] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/ed40131c6805482980bf9e556cef0ce4
I20260809 15:13:53.290390 20948 ts_tablet_manager.cc:1939] T ed40131c6805482980bf9e556cef0ce4 P 66a5f5883e3f4cb6b2155239864c7269: Deleting consensus metadata
I20260809 15:13:53.290904 20950 ts_tablet_manager.cc:1939] T ed40131c6805482980bf9e556cef0ce4 P 855dcf059ea847f3a8ff5ed15c52e113: Deleting consensus metadata
I20260809 15:13:53.292287 20401 catalog_manager.cc:4841] TS 66a5f5883e3f4cb6b2155239864c7269 (127.15.11.131:41929): tablet ed40131c6805482980bf9e556cef0ce4 (table test-table [id=4bd24bcbecde486da28bc4c786f42124]) successfully deleted
I20260809 15:13:53.292667 20403 catalog_manager.cc:4841] TS 855dcf059ea847f3a8ff5ed15c52e113 (127.15.11.130:39259): tablet ed40131c6805482980bf9e556cef0ce4 (table test-table [id=4bd24bcbecde486da28bc4c786f42124]) successfully deleted
I20260809 15:13:53.368028 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 20480
I20260809 15:13:53.397003 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 20620
I20260809 15:13:53.400383 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.397839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.15.11.130:39259)
0809 15:13:53.398047 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.398053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.398075 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:13:53.398183 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.398188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.398283 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.398404 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.399390 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.399398 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.399403 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.399430 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.399480 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.399512 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.399676 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.399769 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.399844 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 15:13:53.401189 20753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:53.490677 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 20620
I20260809 15:13:53.518925 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 20761
I20260809 15:13:53.522280 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:53.519968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56760 (local address 127.15.11.131:41929)
0809 15:13:53.520196 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:53.520200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:53.520218 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:13:53.520385 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:53.520394 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:53.520445 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:53.520522 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:53.521551 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:53.521556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:53.521559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:53.521588 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:53.521621 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:53.521633 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:53.521729 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:53.521831 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 15:13:53.521874 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:13:53.523125 20894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:53.642639 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 20761
I20260809 15:13:53.667191 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 20385
I20260809 15:13:53.668402 20449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:53.776902 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 20385
2026-08-09T15:13:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2674 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 15:13:53.822007 15406 mini_oidc.cc:152] Starting JWKS server
I20260809 15:13:53.824894 15406 webserver.cc:460] Webserver started at https://127.0.0.1:44143/ using document root <none> and password file <none>
I20260809 15:13:53.825187 15406 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 15:13:53.825726 15406 webserver.cc:460] Webserver started at http://127.0.0.1:45427/ using document root <none> and password file <none>
2026-08-09T15:13:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:53Z Disabled control of system clock
I20260809 15:13:53.856051 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37941
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:33627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:37941
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44143/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/testchainca.cert with env {}
W20260809 15:13:54.183835 20961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:54.184226 20961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:54.184427 20961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:54.191689 20961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:54.191852 20961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:54.191979 20961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:54.192113 20961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:54.192231 20961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:13:54.192344 20961 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:13:54.192432 20961 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44143/jwks.json
I20260809 15:13:54.202140 20961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:33627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:37941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44143/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:54.203536 20961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:54.204957 20961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:54.217154 20967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:54.217216 20966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:54.217115 20969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:54.217895 20961 server_base.cc:1034] running on GCE node
I20260809 15:13:54.218567 20961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:54.219847 20961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:54.221102 20961 hybrid_clock.cc:648] HybridClock initialized: now 1786288434221043 us; error 78 us; skew 500 ppm
I20260809 15:13:54.221489 20961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:13:54.223737 20961 webserver.cc:460] Webserver started at http://127.15.11.190:34321/ using document root <none> and password file <none>
I20260809 15:13:54.224506 20961 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:54.224642 20961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:54.224957 20961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:54.227834 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "2ac4aef169214a5683b3885baa00f0f2"
format_stamp: "Formatted at 2026-08-09 15:13:54 on dist-test-slave-8w9v"
server_key: "13fd59ed665d5ffaf33ba5b956d8a976"
server_key_iv: "c677643f09f676cf8f16af08ef7d45a2"
server_key_version: "encryptionkey@0"
I20260809 15:13:54.228750 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "2ac4aef169214a5683b3885baa00f0f2"
format_stamp: "Formatted at 2026-08-09 15:13:54 on dist-test-slave-8w9v"
server_key: "13fd59ed665d5ffaf33ba5b956d8a976"
server_key_iv: "c677643f09f676cf8f16af08ef7d45a2"
server_key_version: "encryptionkey@0"
I20260809 15:13:54.233749 20961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:13:54.237115 20975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:54.238395 20961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:54.238586 20961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "2ac4aef169214a5683b3885baa00f0f2"
format_stamp: "Formatted at 2026-08-09 15:13:54 on dist-test-slave-8w9v"
server_key: "13fd59ed665d5ffaf33ba5b956d8a976"
server_key_iv: "c677643f09f676cf8f16af08ef7d45a2"
server_key_version: "encryptionkey@0"
I20260809 15:13:54.238799 20961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:54.265913 20961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:54.266707 20961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:54.267091 20961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:54.285776 20961 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:37941
I20260809 15:13:54.285796 21032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:37941 every 8 connection(s)
I20260809 15:13:54.287077 20961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:54.290671 21033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:54.294404 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 20961
I20260809 15:13:54.294638 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:54.295439 15406 external_mini_cluster.cc:1428] Setting key 39d773c74c7775d0d9118f937cf2835c
I20260809 15:13:54.304083 21033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2: Bootstrap starting.
I20260809 15:13:54.307853 21033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:54.309172 21033 log.cc:824] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:54.310087 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:54.297454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.15.11.190:37941)
0809 15:13:54.298448 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:54.298458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:54.298498 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:13:54.300068 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:54.300090 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:54.300627 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:54.300958 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:54.301701 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:54.303099 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:54.305034 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:54.305790 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:54.305945 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:54.307278 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:54.307286 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:54.307299 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:54.307873 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:54.308061 (+ 188us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:54.308087 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:54.308288 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:54.308523 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 15:13:54.308882 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":388,"threads_started":1}
I20260809 15:13:54.311967 21033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2: No bootstrap required, opened a new log
I20260809 15:13:54.314003 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:13:54.319607 21033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4aef169214a5683b3885baa00f0f2" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37941 } }
I20260809 15:13:54.319989 21033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:54.320101 21033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac4aef169214a5683b3885baa00f0f2, State: Initialized, Role: FOLLOWER
I20260809 15:13:54.320747 21033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4aef169214a5683b3885baa00f0f2" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37941 } }
I20260809 15:13:54.321026 21033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:54.321175 21033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:54.321390 21033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:54.323678 21033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4aef169214a5683b3885baa00f0f2" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37941 } }
I20260809 15:13:54.324298 21033 leader_election.cc:302] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ac4aef169214a5683b3885baa00f0f2; no voters:
I20260809 15:13:54.324877 21033 leader_election.cc:288] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:54.325035 21038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:54.325759 21038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [term 1 LEADER]: Becoming Leader. State: Replica: 2ac4aef169214a5683b3885baa00f0f2, State: Running, Role: LEADER
I20260809 15:13:54.326273 21038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4aef169214a5683b3885baa00f0f2" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37941 } }
I20260809 15:13:54.326915 21033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:54.329245 21039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ac4aef169214a5683b3885baa00f0f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4aef169214a5683b3885baa00f0f2" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37941 } } }
I20260809 15:13:54.329778 21039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:54.330004 21040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ac4aef169214a5683b3885baa00f0f2. Latest consensus state: current_term: 1 leader_uuid: "2ac4aef169214a5683b3885baa00f0f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac4aef169214a5683b3885baa00f0f2" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37941 } } }
I20260809 15:13:54.330440 21040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:54.333564 21045 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:54.338768 21045 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:54.347666 21045 catalog_manager.cc:1347] Generated new cluster ID: 47472af4d04745a2ac3d9376849f4017
I20260809 15:13:54.347772 21045 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:54.359144 21045 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:54.360402 21045 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:54.371194 21045 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ac4aef169214a5683b3885baa00f0f2: Generated new TSK 0
I20260809 15:13:54.372006 21045 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 15:13:54.410687 21064 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 15:13:54.410948 21064 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 15:13:54.413380 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:54.389898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42872 (local address 127.15.11.190:37941)
0809 15:13:54.390102 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:54.390111 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:54.390576 (+ 465us) server_negotiation.cc:407] Connection header received
0809 15:13:54.391027 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:54.391031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:54.391099 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:54.391257 (+ 158us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:13:54.391841 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:54.392651 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:54.394065 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:54.394504 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:54.394660 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:54.395218 (+ 558us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:13:54.412607 (+ 17389us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 15:13:54.413049 (+ 442us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42872: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 15:13:54.413518 21036 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42872: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 15:13:54.414917 21063 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:54.389818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37941 (local address 127.0.0.1:42872)
0809 15:13:54.390462 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:54.390478 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:54.390680 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:54.391286 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:54.391291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:54.391308 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:13:54.391678 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:54.391688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:54.392789 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:54.392794 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:54.393916 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:54.393926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:54.394726 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:54.394733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:54.394877 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:54.394939 (+ 62us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:13:54.412948 (+ 18009us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 15:13:54.413299 (+ 351us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.11.190:37941: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":329,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.11.190:37941: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44143/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 15:13:54.417191 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 20961
I20260809 15:13:54.418358 21027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:54.512993 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 20961
2026-08-09T15:13:54Z chronyd exiting
I20260809 15:13:54.567641 15406 test_util.cc:175] -----------------------------------------------
I20260809 15:13:54.567778 15406 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786288392170773-15406-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (750 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 15:13:54.571537 15406 mini_oidc.cc:152] Starting JWKS server
I20260809 15:13:54.573491 15406 webserver.cc:460] Webserver started at https://127.0.0.1:35259/ using document root <none> and password file <none>
I20260809 15:13:54.573752 15406 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 15:13:54.574409 15406 webserver.cc:460] Webserver started at http://127.0.0.1:38045/ using document root <none> and password file <none>
2026-08-09T15:13:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:54Z Disabled control of system clock
I20260809 15:13:54.605197 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:46753
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:45143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:46753
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35259/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/testchainca.cert with env {}
W20260809 15:13:54.929277 21072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:54.929651 21072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:54.929837 21072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:54.937033 21072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:54.937175 21072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:54.937297 21072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:54.937405 21072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:54.937508 21072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:13:54.937600 21072 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:13:54.937680 21072 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35259/jwks.json
I20260809 15:13:54.947269 21072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:45143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:46753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:46753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35259/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:54.948555 21072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:54.949846 21072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:54.961637 21077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:54.961704 21080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:54.961767 21078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:54.962430 21072 server_base.cc:1034] running on GCE node
I20260809 15:13:54.963256 21072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:54.964789 21072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:54.966064 21072 hybrid_clock.cc:648] HybridClock initialized: now 1786288434966006 us; error 45 us; skew 500 ppm
I20260809 15:13:54.966420 21072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:13:54.972381 21072 webserver.cc:460] Webserver started at http://127.15.11.190:39431/ using document root <none> and password file <none>
I20260809 15:13:54.973070 21072 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:54.973214 21072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:54.973553 21072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:54.976818 21072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1"
format_stamp: "Formatted at 2026-08-09 15:13:54 on dist-test-slave-8w9v"
server_key: "0e4c27a1032f1dcef21948b716a72958"
server_key_iv: "6eb1db62191b25ee09dee1299de91973"
server_key_version: "encryptionkey@0"
I20260809 15:13:54.977620 21072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1"
format_stamp: "Formatted at 2026-08-09 15:13:54 on dist-test-slave-8w9v"
server_key: "0e4c27a1032f1dcef21948b716a72958"
server_key_iv: "6eb1db62191b25ee09dee1299de91973"
server_key_version: "encryptionkey@0"
I20260809 15:13:54.982470 21072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:13:54.985899 21086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:54.987154 21072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:13:54.987363 21072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1"
format_stamp: "Formatted at 2026-08-09 15:13:54 on dist-test-slave-8w9v"
server_key: "0e4c27a1032f1dcef21948b716a72958"
server_key_iv: "6eb1db62191b25ee09dee1299de91973"
server_key_version: "encryptionkey@0"
I20260809 15:13:54.987607 21072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:55.014601 21072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:55.015511 21072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:55.015825 21072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:55.035097 21072 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:46753
I20260809 15:13:55.035116 21147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:46753 every 8 connection(s)
I20260809 15:13:55.036677 21072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:55.040423 21148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:55.046427 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 21072
I20260809 15:13:55.046628 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:55.047539 15406 external_mini_cluster.cc:1428] Setting key 24660d8b290537e4d833629d3c8d0372
I20260809 15:13:55.051618 21148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1: Bootstrap starting.
I20260809 15:13:55.054831 21148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:55.055873 21148 log.cc:824] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:55.059296 21148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1: No bootstrap required, opened a new log
I20260809 15:13:55.061908 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:55.049545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43644 (local address 127.15.11.190:46753)
0809 15:13:55.050541 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:55.050551 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:55.050593 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:13:55.052416 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:55.052438 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:55.052880 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:55.053289 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:55.053869 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.055204 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.056835 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.057737 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.057934 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:55.059038 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:55.059049 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:55.059066 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:55.059823 (+ 757us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:55.059940 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:55.059967 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:55.060081 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:55.060363 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 15:13:55.060675 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":381,"threads_started":1}
I20260809 15:13:55.065788 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:13:55.068333 21148 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46753 } }
I20260809 15:13:55.068704 21148 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:55.068871 21148 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd0a03e82b2245de9bed6e2e9d31b9f1, State: Initialized, Role: FOLLOWER
I20260809 15:13:55.069443 21148 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46753 } }
I20260809 15:13:55.069693 21148 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:55.069847 21148 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:55.070034 21148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:55.072065 21148 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46753 } }
I20260809 15:13:55.072613 21148 leader_election.cc:302] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd0a03e82b2245de9bed6e2e9d31b9f1; no voters:
I20260809 15:13:55.073175 21148 leader_election.cc:288] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:55.073340 21154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:55.074065 21154 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [term 1 LEADER]: Becoming Leader. State: Replica: fd0a03e82b2245de9bed6e2e9d31b9f1, State: Running, Role: LEADER
I20260809 15:13:55.074697 21154 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46753 } }
I20260809 15:13:55.075403 21148 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:55.078259 21156 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd0a03e82b2245de9bed6e2e9d31b9f1. Latest consensus state: current_term: 1 leader_uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46753 } } }
I20260809 15:13:55.078682 21156 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:55.078503 21155 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd0a03e82b2245de9bed6e2e9d31b9f1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 46753 } } }
I20260809 15:13:55.078925 21155 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:55.080825 21162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:55.085850 21162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:55.094151 21162 catalog_manager.cc:1347] Generated new cluster ID: e86ef5f4a02645f893fc4a0c1dc74361
I20260809 15:13:55.094273 21162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:55.107607 21162 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:55.109438 21162 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:55.123292 21162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd0a03e82b2245de9bed6e2e9d31b9f1: Generated new TSK 0
I20260809 15:13:55.124311 21162 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 15:13:55.168823 21181 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 15:13:55.168980 21181 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 15:13:55.170068 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:55.143157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.15.11.190:46753)
0809 15:13:55.143375 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:55.143384 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:55.143768 (+ 384us) server_negotiation.cc:407] Connection header received
0809 15:13:55.144011 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:55.144015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:55.144092 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:55.144302 (+ 210us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:13:55.144818 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.145523 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.146895 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.147332 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.147504 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:55.147984 (+ 480us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:13:55.169344 (+ 21360us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 15:13:55.169762 (+ 418us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 15:13:55.170217 21151 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 15:13:55.170231 21179 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:55.143048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:46753 (local address 127.0.0.1:43648)
0809 15:13:55.143663 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:55.143677 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:55.143839 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:55.144317 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:55.144322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:55.144333 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:13:55.144617 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:55.144627 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.145667 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.145673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:55.146727 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:55.146738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.147521 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.147527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:55.147683 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:55.147714 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:13:55.169579 (+ 21865us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 15:13:55.169835 (+ 256us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.11.190:46753: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35259/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":340,"threads_started":1}
I20260809 15:13:55.171542 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 21072
I20260809 15:13:55.172780 21143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:55.265307 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 21072
2026-08-09T15:13:55Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (753 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 15:13:55.325116 15406 mini_oidc.cc:152] Starting JWKS server
I20260809 15:13:55.326931 15406 webserver.cc:460] Webserver started at https://127.0.0.1:41837/ using document root <none> and password file <none>
I20260809 15:13:55.327122 15406 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 15:13:55.327679 15406 webserver.cc:460] Webserver started at http://127.0.0.1:45247/ using document root <none> and password file <none>
2026-08-09T15:13:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:55Z Disabled control of system clock
I20260809 15:13:55.358211 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:36887
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:40383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:36887
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41837/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 15:13:55.688808 21189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:55.689085 21189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:55.689284 21189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:55.696368 21189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:55.696494 21189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:55.696564 21189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:55.696655 21189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:55.696736 21189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:13:55.696830 21189 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:13:55.696908 21189 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41837/jwks.json
I20260809 15:13:55.705696 21189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:36887
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:36887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41837/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:55.707005 21189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:55.708839 21189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:55.720927 21194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:55.720808 21197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:55.720875 21195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:55.721962 21189 server_base.cc:1034] running on GCE node
I20260809 15:13:55.722718 21189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:55.724211 21189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:55.725466 21189 hybrid_clock.cc:648] HybridClock initialized: now 1786288435725421 us; error 32 us; skew 500 ppm
I20260809 15:13:55.725831 21189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:13:55.728245 21189 webserver.cc:460] Webserver started at http://127.15.11.190:37657/ using document root <none> and password file <none>
I20260809 15:13:55.728966 21189 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:55.729084 21189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:55.729422 21189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:55.732514 21189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "e9fedadc0a89489290a8d9b68a7168cd"
format_stamp: "Formatted at 2026-08-09 15:13:55 on dist-test-slave-8w9v"
server_key: "61351fc05dafb58514a39fa1b6f438d8"
server_key_iv: "26ad4ae75c779600290e88041a760214"
server_key_version: "encryptionkey@0"
I20260809 15:13:55.733438 21189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "e9fedadc0a89489290a8d9b68a7168cd"
format_stamp: "Formatted at 2026-08-09 15:13:55 on dist-test-slave-8w9v"
server_key: "61351fc05dafb58514a39fa1b6f438d8"
server_key_iv: "26ad4ae75c779600290e88041a760214"
server_key_version: "encryptionkey@0"
I20260809 15:13:55.738922 21189 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:13:55.742550 21203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:55.743873 21189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 15:13:55.744102 21189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "e9fedadc0a89489290a8d9b68a7168cd"
format_stamp: "Formatted at 2026-08-09 15:13:55 on dist-test-slave-8w9v"
server_key: "61351fc05dafb58514a39fa1b6f438d8"
server_key_iv: "26ad4ae75c779600290e88041a760214"
server_key_version: "encryptionkey@0"
I20260809 15:13:55.744445 21189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:55.767216 21189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:55.767992 21189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:55.768332 21189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:55.786957 21189 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:36887
I20260809 15:13:55.786994 21260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:36887 every 8 connection(s)
I20260809 15:13:55.788275 21189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:55.791913 21261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:55.797940 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 21189
I20260809 15:13:55.798187 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:55.799031 15406 external_mini_cluster.cc:1428] Setting key 4b1f35ea77859faf3e89b58b9cde12f2
I20260809 15:13:55.803444 21261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd: Bootstrap starting.
I20260809 15:13:55.806550 21261 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:55.807545 21261 log.cc:824] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:55.810286 21261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd: No bootstrap required, opened a new log
I20260809 15:13:55.812358 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:55.801175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.15.11.190:36887)
0809 15:13:55.802258 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:55.802268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:55.802307 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:13:55.803978 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:55.804019 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:55.804514 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:55.804930 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:55.805449 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.806691 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.808082 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.808772 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.808989 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:55.810115 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:55.810122 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:55.810136 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:55.810691 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:55.810768 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:55.810791 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:55.810936 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:55.811132 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 15:13:55.811407 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":449,"threads_started":1}
I20260809 15:13:55.816310 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:13:55.818822 21261 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9fedadc0a89489290a8d9b68a7168cd" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36887 } }
I20260809 15:13:55.819243 21261 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:55.819357 21261 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9fedadc0a89489290a8d9b68a7168cd, State: Initialized, Role: FOLLOWER
I20260809 15:13:55.819922 21261 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9fedadc0a89489290a8d9b68a7168cd" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36887 } }
I20260809 15:13:55.820189 21261 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:55.820356 21261 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:55.820538 21261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:55.822625 21261 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9fedadc0a89489290a8d9b68a7168cd" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36887 } }
I20260809 15:13:55.823205 21261 leader_election.cc:302] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9fedadc0a89489290a8d9b68a7168cd; no voters:
I20260809 15:13:55.823910 21261 leader_election.cc:288] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:55.824043 21268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:55.824831 21268 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [term 1 LEADER]: Becoming Leader. State: Replica: e9fedadc0a89489290a8d9b68a7168cd, State: Running, Role: LEADER
I20260809 15:13:55.825297 21268 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9fedadc0a89489290a8d9b68a7168cd" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36887 } }
I20260809 15:13:55.826237 21261 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:55.828450 21269 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9fedadc0a89489290a8d9b68a7168cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9fedadc0a89489290a8d9b68a7168cd" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36887 } } }
I20260809 15:13:55.828816 21269 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:55.828744 21270 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9fedadc0a89489290a8d9b68a7168cd. Latest consensus state: current_term: 1 leader_uuid: "e9fedadc0a89489290a8d9b68a7168cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9fedadc0a89489290a8d9b68a7168cd" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36887 } } }
I20260809 15:13:55.829169 21270 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:55.831108 21274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:55.836576 21274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:55.845321 21274 catalog_manager.cc:1347] Generated new cluster ID: 962988d18e804874b2c2f0ab33812177
I20260809 15:13:55.845427 21274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:55.865526 21274 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:55.867322 21274 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:55.880106 21274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9fedadc0a89489290a8d9b68a7168cd: Generated new TSK 0
I20260809 15:13:55.881078 21274 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:13:55.907550 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:55.899592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.15.11.190:36887)
0809 15:13:55.899822 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:55.899831 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:55.900282 (+ 451us) server_negotiation.cc:407] Connection header received
0809 15:13:55.900507 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:55.900511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:55.900575 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:55.900688 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:13:55.901215 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.901917 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.903898 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.904406 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.904503 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:55.905096 (+ 593us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:13:55.906887 (+ 1791us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 15:13:55.907218 (+ 331us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59694: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 15:13:55.907680 21266 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59694: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 15:13:55.907711 21293 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:55.899521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:36887 (local address 127.0.0.1:59694)
0809 15:13:55.900180 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:55.900199 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:55.900339 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:55.900735 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:55.900741 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:55.900754 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:13:55.901073 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:55.901084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.902065 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.902070 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:55.903739 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:55.903755 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:55.904609 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:55.904616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:55.904812 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:55.904859 (+ 47us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:13:55.907083 (+ 2224us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 15:13:55.907314 (+ 231us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.11.190:36887: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41837/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":330,"threads_started":1}
I20260809 15:13:55.908883 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 21189
I20260809 15:13:55.909999 21255 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 15:13:55.919770 21294 webserver.cc:533] Webserver: sslize failed:
E20260809 15:13:55.919857 21294 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 15:13:56.000247 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 21189
2026-08-09T15:13:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (734 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T15:13:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:56Z Disabled control of system clock
I20260809 15:13:56.090626 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:34269
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:37425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:34269
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/testchainca.cert with env {}
W20260809 15:13:56.418973 21299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:56.419390 21299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:56.419580 21299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:56.426939 21299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:56.427089 21299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:56.427160 21299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:56.427276 21299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:56.427371 21299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:13:56.427481 21299 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:13:56.427537 21299 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 15:13:56.437629 21299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:34269
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:34269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:56.439163 21299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:56.440652 21299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:56.453320 21309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:56.453284 21306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:56.453290 21307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:56.454174 21299 server_base.cc:1034] running on GCE node
I20260809 15:13:56.455049 21299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:56.456593 21299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:56.457845 21299 hybrid_clock.cc:648] HybridClock initialized: now 1786288436457800 us; error 58 us; skew 500 ppm
I20260809 15:13:56.458225 21299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:13:56.460834 21299 webserver.cc:460] Webserver started at http://127.15.11.190:42809/ using document root <none> and password file <none>
I20260809 15:13:56.461655 21299 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:56.461822 21299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:56.462168 21299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:56.465345 21299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "b97a1005ab024c308b3cbb30bee17902"
format_stamp: "Formatted at 2026-08-09 15:13:56 on dist-test-slave-8w9v"
server_key: "cdf88d0780706f039eb05de083452ad2"
server_key_iv: "bf6c6a5589bc65f7a274c9c86a31ceee"
server_key_version: "encryptionkey@0"
I20260809 15:13:56.466228 21299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "b97a1005ab024c308b3cbb30bee17902"
format_stamp: "Formatted at 2026-08-09 15:13:56 on dist-test-slave-8w9v"
server_key: "cdf88d0780706f039eb05de083452ad2"
server_key_iv: "bf6c6a5589bc65f7a274c9c86a31ceee"
server_key_version: "encryptionkey@0"
I20260809 15:13:56.471988 21299 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 15:13:56.475807 21315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:56.477263 21299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:13:56.477485 21299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "b97a1005ab024c308b3cbb30bee17902"
format_stamp: "Formatted at 2026-08-09 15:13:56 on dist-test-slave-8w9v"
server_key: "cdf88d0780706f039eb05de083452ad2"
server_key_iv: "bf6c6a5589bc65f7a274c9c86a31ceee"
server_key_version: "encryptionkey@0"
I20260809 15:13:56.477720 21299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:56.494424 21299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:56.495253 21299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:56.495549 21299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:56.515107 21299 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:34269
I20260809 15:13:56.515132 21368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:34269 every 8 connection(s)
I20260809 15:13:56.516628 21299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:56.520478 21369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:56.521517 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 21299
I20260809 15:13:56.521715 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:56.522612 15406 external_mini_cluster.cc:1428] Setting key e7d2a72daa5a4529b49a77caa96f00f8
I20260809 15:13:56.533573 21369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902: Bootstrap starting.
I20260809 15:13:56.536077 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:56.524727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.15.11.190:34269)
0809 15:13:56.525722 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:56.525732 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:56.525780 (+ 48us) server_negotiation.cc:407] Connection header received
0809 15:13:56.527458 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:56.527480 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:56.527926 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:56.528382 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:56.528919 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:56.530136 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:56.531399 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:56.532076 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:56.532315 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:56.533758 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:56.533767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:56.533780 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:13:56.534346 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:56.534441 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:13:56.534462 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:56.534611 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:56.534826 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 15:13:56.535148 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":339,"threads_started":1}
I20260809 15:13:56.537086 21369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:56.538179 21369 log.cc:824] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:56.539960 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:13:56.541675 21369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902: No bootstrap required, opened a new log
I20260809 15:13:56.549378 21369 raft_consensus.cc:348] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b97a1005ab024c308b3cbb30bee17902" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34269 } }
I20260809 15:13:56.549849 21369 raft_consensus.cc:374] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:56.549968 21369 raft_consensus.cc:729] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b97a1005ab024c308b3cbb30bee17902, State: Initialized, Role: FOLLOWER
I20260809 15:13:56.550549 21369 consensus_queue.cc:260] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b97a1005ab024c308b3cbb30bee17902" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34269 } }
I20260809 15:13:56.550793 21369 raft_consensus.cc:388] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:56.550957 21369 raft_consensus.cc:482] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:56.551146 21369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:56.553434 21369 raft_consensus.cc:504] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b97a1005ab024c308b3cbb30bee17902" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34269 } }
I20260809 15:13:56.553942 21369 leader_election.cc:302] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b97a1005ab024c308b3cbb30bee17902; no voters:
I20260809 15:13:56.554541 21369 leader_election.cc:288] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:56.554713 21374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:56.555461 21374 raft_consensus.cc:686] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [term 1 LEADER]: Becoming Leader. State: Replica: b97a1005ab024c308b3cbb30bee17902, State: Running, Role: LEADER
I20260809 15:13:56.555956 21374 consensus_queue.cc:237] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b97a1005ab024c308b3cbb30bee17902" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34269 } }
I20260809 15:13:56.556809 21369 sys_catalog.cc:564] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:56.559703 21376 sys_catalog.cc:455] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b97a1005ab024c308b3cbb30bee17902. Latest consensus state: current_term: 1 leader_uuid: "b97a1005ab024c308b3cbb30bee17902" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b97a1005ab024c308b3cbb30bee17902" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34269 } } }
I20260809 15:13:56.560062 21376 sys_catalog.cc:458] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:56.559818 21375 sys_catalog.cc:455] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b97a1005ab024c308b3cbb30bee17902" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b97a1005ab024c308b3cbb30bee17902" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34269 } } }
I20260809 15:13:56.560271 21375 sys_catalog.cc:458] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:56.564702 21380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:56.569674 21380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:56.577443 21380 catalog_manager.cc:1347] Generated new cluster ID: 752a450826a944fb98eeaa7109fb84ec
I20260809 15:13:56.577543 21380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:56.598162 21380 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:56.599965 21380 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:56.609539 21380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b97a1005ab024c308b3cbb30bee17902: Generated new TSK 0
I20260809 15:13:56.610587 21380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:13:56.699771 21372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:56.634942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56722 (local address 127.15.11.190:34269)
0809 15:13:56.635167 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:56.635178 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:56.635560 (+ 382us) server_negotiation.cc:407] Connection header received
0809 15:13:56.635811 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:56.635815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:56.635882 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:56.636048 (+ 166us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:13:56.636632 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:56.637385 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:56.639405 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:56.639877 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:56.639990 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:56.640663 (+ 673us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:13:56.699182 (+ 58519us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 15:13:56.699533 (+ 351us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56722: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 15:13:56.699951 21372 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56722: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 15:13:56.700276 21401 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:13:56.634737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:34269 (local address 127.0.0.1:56722)
0809 15:13:56.635443 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 15:13:56.635464 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:13:56.635625 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:13:56.636102 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:13:56.636107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:13:56.636124 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:13:56.636475 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:56.636488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:56.637541 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:56.637547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:56.639255 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:13:56.639269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:13:56.640095 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:13:56.640102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:13:56.640304 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:56.640349 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:13:56.699499 (+ 59150us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 15:13:56.699824 (+ 325us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.11.190:34269: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":358,"threads_started":1}
I20260809 15:13:56.702268 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 21299
I20260809 15:13:56.703562 21364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:13:56.808956 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 21299
2026-08-09T15:13:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (789 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (421 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (434 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:13:57 dist-test-slave-8w9v krb5kdc[21418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:13:57 dist-test-slave-8w9v krb5kdc[21418](info): set up 2 sockets
Aug 09 15:13:57 dist-test-slave-8w9v krb5kdc[21418](info): commencing operation
krb5kdc: starting...
W20260809 15:13:58.783669 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:13:58 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:13:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:13:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:13:59.003926 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:34023
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:34023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:59.324968 21432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:59.325225 21432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:59.325318 21432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:59.331874 21432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:13:59.331965 21432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:59.332015 21432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:59.332060 21432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:13:59.332093 21432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:13:59.341457 21432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:34023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:34023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:59.342571 21432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:59.343804 21432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:59.355507 21440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:59.355559 21437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:59.355652 21438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:59.356710 21432 server_base.cc:1034] running on GCE node
I20260809 15:13:59.357451 21432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:59.358661 21432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:59.359867 21432 hybrid_clock.cc:648] HybridClock initialized: now 1786288439359807 us; error 50 us; skew 500 ppm
I20260809 15:13:59.360289 21432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:59 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288439, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:59.363458 21432 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:13:59.364938 21432 webserver.cc:460] Webserver started at http://127.15.11.190:35265/ using document root <none> and password file <none>
I20260809 15:13:59.365628 21432 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:59.365770 21432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:59.366112 21432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:59.368988 21432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "c89f990d675e49c9a16a98ad092b7d89"
format_stamp: "Formatted at 2026-08-09 15:13:59 on dist-test-slave-8w9v"
server_key: "37a4b70e0c14ae3c66ed576088f4e7cc"
server_key_iv: "dd0802df24490327c38d09849d24ce02"
server_key_version: "encryptionkey@0"
I20260809 15:13:59.369700 21432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "c89f990d675e49c9a16a98ad092b7d89"
format_stamp: "Formatted at 2026-08-09 15:13:59 on dist-test-slave-8w9v"
server_key: "37a4b70e0c14ae3c66ed576088f4e7cc"
server_key_iv: "dd0802df24490327c38d09849d24ce02"
server_key_version: "encryptionkey@0"
I20260809 15:13:59.374727 21432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:13:59.378182 21447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:59.379493 21432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:13:59.379685 21432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "c89f990d675e49c9a16a98ad092b7d89"
format_stamp: "Formatted at 2026-08-09 15:13:59 on dist-test-slave-8w9v"
server_key: "37a4b70e0c14ae3c66ed576088f4e7cc"
server_key_iv: "dd0802df24490327c38d09849d24ce02"
server_key_version: "encryptionkey@0"
I20260809 15:13:59.379884 21432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:13:59.391162 21432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:13:59.394923 21432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:13:59.395200 21432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:13:59.413419 21432 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:34023
I20260809 15:13:59.413462 21501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:34023 every 8 connection(s)
I20260809 15:13:59.414770 21432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:13:59.418406 21502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:13:59.425103 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 21432
I20260809 15:13:59.425338 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:13:59.426152 15406 external_mini_cluster.cc:1428] Setting key 1d8e9d24263e84164cc77d4aa2decde6
I20260809 15:13:59.429505 21502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89: Bootstrap starting.
I20260809 15:13:59.432821 21502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89: Neither blocks nor log segments found. Creating new log.
I20260809 15:13:59.434224 21502 log.cc:824] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89: Log is configured to *not* fsync() on all Append() calls
I20260809 15:13:59.437072 21502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89: No bootstrap required, opened a new log
Aug 09 15:13:59 dist-test-slave-8w9v krb5kdc[21418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:13:59.444689 21502 raft_consensus.cc:348] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c89f990d675e49c9a16a98ad092b7d89" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34023 } }
I20260809 15:13:59.445070 21502 raft_consensus.cc:374] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:13:59.445212 21502 raft_consensus.cc:729] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c89f990d675e49c9a16a98ad092b7d89, State: Initialized, Role: FOLLOWER
I20260809 15:13:59.445813 21502 consensus_queue.cc:260] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c89f990d675e49c9a16a98ad092b7d89" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34023 } }
I20260809 15:13:59.446074 21502 raft_consensus.cc:388] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:13:59.446244 21502 raft_consensus.cc:482] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:13:59.446432 21502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:13:59.449157 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:13:59.428311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.15.11.190:34023)
0809 15:13:59.429028 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0809 15:13:59.429038 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:13:59.429075 (+ 37us) server_negotiation.cc:407] Connection header received
0809 15:13:59.430665 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:13:59.430688 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:13:59.431024 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:13:59.431346 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:13:59.433225 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:59.435170 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:59.436540 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:13:59.437175 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:13:59.437429 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:13:59.441128 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:13:59.441135 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:13:59.441147 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:13:59.441179 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:13:59.444225 (+ 3046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:59.444911 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:59.444919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:59.444929 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:59.445147 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:13:59.445604 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:13:59.445615 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:13:59.445618 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:13:59.446138 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:13:59.446439 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 15:13:59.447463 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0809 15:13:59.447877 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":333,"threads_started":1}
I20260809 15:13:59.450489 21502 raft_consensus.cc:504] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c89f990d675e49c9a16a98ad092b7d89" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34023 } }
I20260809 15:13:59.451018 21502 leader_election.cc:302] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c89f990d675e49c9a16a98ad092b7d89; no voters:
I20260809 15:13:59.451601 21502 leader_election.cc:288] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:13:59.451696 21507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:13:59.452312 21507 raft_consensus.cc:686] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [term 1 LEADER]: Becoming Leader. State: Replica: c89f990d675e49c9a16a98ad092b7d89, State: Running, Role: LEADER
I20260809 15:13:59.452963 21507 consensus_queue.cc:237] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c89f990d675e49c9a16a98ad092b7d89" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34023 } }
I20260809 15:13:59.453909 21502 sys_catalog.cc:564] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:13:59.455794 21508 sys_catalog.cc:455] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c89f990d675e49c9a16a98ad092b7d89. Latest consensus state: current_term: 1 leader_uuid: "c89f990d675e49c9a16a98ad092b7d89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c89f990d675e49c9a16a98ad092b7d89" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34023 } } }
I20260809 15:13:59.456307 21508 sys_catalog.cc:458] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:59.455788 21509 sys_catalog.cc:455] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c89f990d675e49c9a16a98ad092b7d89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c89f990d675e49c9a16a98ad092b7d89" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34023 } } }
I20260809 15:13:59.456895 21509 sys_catalog.cc:458] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89 [sys.catalog]: This master's current role is: LEADER
I20260809 15:13:59.460891 21513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:13:59.467237 21513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:13:59.475808 21513 catalog_manager.cc:1347] Generated new cluster ID: 681e9568d6694edc966b02abc014f00f
I20260809 15:13:59.475936 21513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:13:59.492959 21513 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:13:59.494560 21513 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:13:59.503235 21513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c89f990d675e49c9a16a98ad092b7d89: Generated new TSK 0
I20260809 15:13:59.504050 21513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:13:59.571836 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:34023
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:13:59.917465 21529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:13:59.917742 21529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:13:59.917936 21529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:13:59.924854 21529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:13:59.924973 21529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:13:59.925153 21529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:13:59.934535 21529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:34023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:13:59.935779 21529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:13:59.937143 21529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:13:59.950520 21536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:59.950639 21535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:13:59.950587 21538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:13:59.951591 21529 server_base.cc:1034] running on GCE node
I20260809 15:13:59.952248 21529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:13:59.953135 21529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:13:59.954378 21529 hybrid_clock.cc:648] HybridClock initialized: now 1786288439954303 us; error 61 us; skew 500 ppm
I20260809 15:13:59.954733 21529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:13:59 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288439, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:13:59.958029 21529 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:13:59.959390 21529 webserver.cc:460] Webserver started at http://127.15.11.129:44155/ using document root <none> and password file <none>
I20260809 15:13:59.960062 21529 fs_manager.cc:359] Metadata directory not provided
I20260809 15:13:59.960260 21529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:13:59.960588 21529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:13:59.963433 21529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "019a0f4adcaa427aa51fcc3a3c4ea613"
format_stamp: "Formatted at 2026-08-09 15:13:59 on dist-test-slave-8w9v"
server_key: "800923685c8a1b8e0b73353f0d305170"
server_key_iv: "aefdf8135ac8d14994a27f5525fb70ed"
server_key_version: "encryptionkey@0"
I20260809 15:13:59.964386 21529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "019a0f4adcaa427aa51fcc3a3c4ea613"
format_stamp: "Formatted at 2026-08-09 15:13:59 on dist-test-slave-8w9v"
server_key: "800923685c8a1b8e0b73353f0d305170"
server_key_iv: "aefdf8135ac8d14994a27f5525fb70ed"
server_key_version: "encryptionkey@0"
I20260809 15:13:59.969475 21529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:13:59.972973 21545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:13:59.974184 21529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:13:59.974375 21529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "019a0f4adcaa427aa51fcc3a3c4ea613"
format_stamp: "Formatted at 2026-08-09 15:13:59 on dist-test-slave-8w9v"
server_key: "800923685c8a1b8e0b73353f0d305170"
server_key_iv: "aefdf8135ac8d14994a27f5525fb70ed"
server_key_version: "encryptionkey@0"
I20260809 15:13:59.974606 21529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:00.010164 21529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:00.014231 21529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:00.014484 21529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:00.015527 21529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:00.016913 21529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:00.016987 21529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:00.017073 21529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:00.017132 21529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:00.051951 21529 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:40207
I20260809 15:14:00.052079 21674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:40207 every 8 connection(s)
I20260809 15:14:00.053324 21529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:14:00.056351 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 21529
I20260809 15:14:00.056593 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:14:00.057555 15406 external_mini_cluster.cc:1428] Setting key aa23094276a031a421591f15271a7b5a
Aug 09 15:14:00 dist-test-slave-8w9v krb5kdc[21418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288439, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:00.077462 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:00.056285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:52855 (local address 127.15.11.190:34023)
0809 15:14:00.056909 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:00.056917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:00.057976 (+ 1059us) server_negotiation.cc:407] Connection header received
0809 15:14:00.060461 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:00.060465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:00.060529 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:00.060619 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:00.062841 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:00.063540 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:00.066171 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:00.066550 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:00.066664 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:00.071051 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:00.071057 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:00.071060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:00.071094 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:00.073894 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:00.074737 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:00.074743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:00.074749 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:00.074892 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:00.075395 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:00.075402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:00.075410 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:00.075680 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:00.075894 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:00.076763 (+ 869us) server_negotiation.cc:299] Negotiation successful
0809 15:14:00.077013 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":312,"threads_started":1}
I20260809 15:14:00.078649 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:00.056861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:34023 (local address 127.15.11.129:52855)
0809 15:14:00.057770 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:00.057831 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:00.060158 (+ 2327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:00.060928 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:00.060942 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:00.061777 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:00.062640 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:00.062670 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:00.063728 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:00.063744 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:00.066023 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:00.066042 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:00.066839 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:00.066849 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:00.067214 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:00.067998 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:00.068036 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:00.070815 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:00.074119 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:00.074128 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:00.074149 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:00.074589 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:00.075025 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:00.075031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:00.075035 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:00.075256 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:00.075834 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:00.075843 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:00.076341 (+ 498us) client_negotiation.cc:769] Sending connection context
0809 15:14:00.076647 (+ 306us) client_negotiation.cc:240] Negotiation successful
0809 15:14:00.077194 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":341,"threads_started":1}
I20260809 15:14:00.080461 21675 heartbeater.cc:347] Connected to a master server at 127.15.11.190:34023
I20260809 15:14:00.080796 21675 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:00.081568 21675 heartbeater.cc:511] Master 127.15.11.190:34023 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
I20260809 15:14:00.083985 21464 ts_manager.cc:194] Registered new tserver with Master: 019a0f4adcaa427aa51fcc3a3c4ea613 (127.15.11.129:40207)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
I20260809 15:14:00.085729 21464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:52855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:14:00.112864 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:34023
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:00.440349 21681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:00.440619 21681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:00.440805 21681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:00.447731 21681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:00.447819 21681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:00.447973 21681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:14:00.457248 21681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:34023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:00.458427 21681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:00.459707 21681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:00.472995 21687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:00.473037 21689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:00.472999 21686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:00.473915 21681 server_base.cc:1034] running on GCE node
I20260809 15:14:00.474545 21681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:00.475481 21681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:00.476681 21681 hybrid_clock.cc:648] HybridClock initialized: now 1786288440476611 us; error 68 us; skew 500 ppm
I20260809 15:14:00.477025 21681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:00 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288440, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:00.480408 21681 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:14:00.481876 21681 webserver.cc:460] Webserver started at http://127.15.11.130:44423/ using document root <none> and password file <none>
I20260809 15:14:00.482635 21681 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:00.482777 21681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:00.483117 21681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:00.486218 21681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "f0a5caa2200640fe9f2618df60b3cf3c"
format_stamp: "Formatted at 2026-08-09 15:14:00 on dist-test-slave-8w9v"
server_key: "74c8b64840dfcd110ad673fa2d209683"
server_key_iv: "0ef463095b4767ebaa2e0c9b332b0257"
server_key_version: "encryptionkey@0"
I20260809 15:14:00.487023 21681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "f0a5caa2200640fe9f2618df60b3cf3c"
format_stamp: "Formatted at 2026-08-09 15:14:00 on dist-test-slave-8w9v"
server_key: "74c8b64840dfcd110ad673fa2d209683"
server_key_iv: "0ef463095b4767ebaa2e0c9b332b0257"
server_key_version: "encryptionkey@0"
I20260809 15:14:00.492502 21681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:14:00.497025 21697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:00.498809 21681 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:14:00.499001 21681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "f0a5caa2200640fe9f2618df60b3cf3c"
format_stamp: "Formatted at 2026-08-09 15:14:00 on dist-test-slave-8w9v"
server_key: "74c8b64840dfcd110ad673fa2d209683"
server_key_iv: "0ef463095b4767ebaa2e0c9b332b0257"
server_key_version: "encryptionkey@0"
I20260809 15:14:00.499254 21681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:00.525651 21681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:00.529963 21681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:00.530193 21681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:00.531129 21681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:00.532539 21681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:00.532616 21681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:00.532717 21681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:00.532763 21681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:00.565857 21681 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:39305
I20260809 15:14:00.565966 21816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:39305 every 8 connection(s)
I20260809 15:14:00.567193 21681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:14:00.575101 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 21681
I20260809 15:14:00.575277 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:14:00.575783 15406 external_mini_cluster.cc:1428] Setting key 5ee29c626af5e73b20fc59d0070abca9
Aug 09 15:14:00 dist-test-slave-8w9v krb5kdc[21418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288440, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:00.588027 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:00.569615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:59365 (local address 127.15.11.190:34023)
0809 15:14:00.569954 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:00.569959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:00.570972 (+ 1013us) server_negotiation.cc:407] Connection header received
0809 15:14:00.572792 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:00.572796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:00.572865 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:00.572952 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:00.574366 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:00.575426 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:00.577327 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:00.577856 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:00.578004 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:00.581938 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:00.581946 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:00.581954 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:00.581998 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:00.585039 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:00.585783 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:00.585793 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:00.585796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:00.585928 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:00.586365 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:00.586369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:00.586377 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:00.586622 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:00.586782 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:00.587450 (+ 668us) server_negotiation.cc:299] Negotiation successful
0809 15:14:00.587703 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260809 15:14:00.588994 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:00.570087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:34023 (local address 127.15.11.130:59365)
0809 15:14:00.570797 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:00.570840 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:00.572540 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:00.573157 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:00.573165 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:00.573637 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:00.574215 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:00.574236 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:00.575562 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:00.575566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:00.577112 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:00.577174 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:00.578097 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:00.578102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:00.578397 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:00.579150 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:00.579185 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:00.581707 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:00.585191 (+ 3484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:00.585200 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:00.585215 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:00.585651 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:00.586047 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:00.586052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:00.586055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:00.586241 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:00.586757 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:00.586765 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:00.587092 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 15:14:00.587344 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 15:14:00.587878 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":289,"threads_started":1}
I20260809 15:14:00.590430 21817 heartbeater.cc:347] Connected to a master server at 127.15.11.190:34023
I20260809 15:14:00.590736 21817 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:00.591428 21817 heartbeater.cc:511] Master 127.15.11.190:34023 requested a full tablet report, sending...
I20260809 15:14:00.592943 21464 ts_manager.cc:194] Registered new tserver with Master: f0a5caa2200640fe9f2618df60b3cf3c (127.15.11.130:39305)
I20260809 15:14:00.594208 21464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:59365
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:14:00.627921 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:34023
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:46849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:00.967335 21822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:00.967598 21822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:00.967689 21822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:00.975267 21822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:00.975375 21822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:00.975499 21822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:14:00.985576 21822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:46849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:34023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:00.986827 21822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:00.988003 21822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:01.001797 21827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:01.001875 21828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:01.002039 21831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:01.002586 21822 server_base.cc:1034] running on GCE node
I20260809 15:14:01.003194 21822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:01.004208 21822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:01.005441 21822 hybrid_clock.cc:648] HybridClock initialized: now 1786288441005368 us; error 40 us; skew 500 ppm
I20260809 15:14:01.005795 21822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:01.009124 21822 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:14:01.010478 21822 webserver.cc:460] Webserver started at http://127.15.11.131:41065/ using document root <none> and password file <none>
I20260809 15:14:01.011111 21822 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:01.011210 21822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:01.011461 21822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:01.014420 21822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "32a6ff06ba64428c8fc7570feed27717"
format_stamp: "Formatted at 2026-08-09 15:14:01 on dist-test-slave-8w9v"
server_key: "f92f64b7d535a69ca9d55686366e0db0"
server_key_iv: "9a60f03d8d27fa10fe49ee5e2ab6c3d4"
server_key_version: "encryptionkey@0"
I20260809 15:14:01.015190 21822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "32a6ff06ba64428c8fc7570feed27717"
format_stamp: "Formatted at 2026-08-09 15:14:01 on dist-test-slave-8w9v"
server_key: "f92f64b7d535a69ca9d55686366e0db0"
server_key_iv: "9a60f03d8d27fa10fe49ee5e2ab6c3d4"
server_key_version: "encryptionkey@0"
I20260809 15:14:01.020556 21822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 15:14:01.023944 21840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:01.025262 21822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:01.025457 21822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "32a6ff06ba64428c8fc7570feed27717"
format_stamp: "Formatted at 2026-08-09 15:14:01 on dist-test-slave-8w9v"
server_key: "f92f64b7d535a69ca9d55686366e0db0"
server_key_iv: "9a60f03d8d27fa10fe49ee5e2ab6c3d4"
server_key_version: "encryptionkey@0"
I20260809 15:14:01.025681 21822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:01.039999 21822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:01.044628 21822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:01.044937 21822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:01.045956 21822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:01.047415 21822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:01.047534 21822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:01.047686 21822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:01.047766 21822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:01.084636 21822 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:35441
I20260809 15:14:01.084666 21960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:35441 every 8 connection(s)
I20260809 15:14:01.086057 21822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:14:01.089440 21675 heartbeater.cc:503] Master 127.15.11.190:34023 was elected leader, sending a full tablet report...
I20260809 15:14:01.090505 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 21822
I20260809 15:14:01.090667 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:14:01.091194 15406 external_mini_cluster.cc:1428] Setting key d3054e9dff1f8cb683ff7cac1c44279a
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:01.107558 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:01.088716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:40321 (local address 127.15.11.190:34023)
0809 15:14:01.089288 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.089293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.090539 (+ 1246us) server_negotiation.cc:407] Connection header received
0809 15:14:01.092750 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.092756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.092852 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.092997 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.094812 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.095655 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.097237 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.097653 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.097767 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:01.101052 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:01.101057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:01.101060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:01.101093 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:01.103951 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.104876 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.104882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.104885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.105081 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.105656 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.105661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.105665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.105942 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:01.106115 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:01.106912 (+ 797us) server_negotiation.cc:299] Negotiation successful
0809 15:14:01.107135 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":261,"threads_started":1}
I20260809 15:14:01.108567 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:01.089331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:34023 (local address 127.15.11.131:40321)
0809 15:14:01.090351 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:01.090410 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:01.092428 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:01.093333 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:01.093352 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:01.093862 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:01.094623 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:01.094643 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.095796 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.095799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:01.097097 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:01.097110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.097845 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.097849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:01.098033 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:01.098632 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:01.098653 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:01.100869 (+ 2216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:01.104084 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:01.104090 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:01.104101 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:01.104716 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:01.105213 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:01.105218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:01.105222 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:01.105496 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:01.106096 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:01.106104 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:01.106469 (+ 365us) client_negotiation.cc:769] Sending connection context
0809 15:14:01.106795 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 15:14:01.107336 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":406,"threads_started":1}
I20260809 15:14:01.109922 21961 heartbeater.cc:347] Connected to a master server at 127.15.11.190:34023
I20260809 15:14:01.110260 21961 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:01.110986 21961 heartbeater.cc:511] Master 127.15.11.190:34023 requested a full tablet report, sending...
I20260809 15:14:01.112453 21464 ts_manager.cc:194] Registered new tserver with Master: 32a6ff06ba64428c8fc7570feed27717 (127.15.11.131:35441)
I20260809 15:14:01.113363 21464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:40321
I20260809 15:14:01.118955 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/test-admin.keytab.
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:01.149974 15406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 15:14:01.155336 15406 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5cc' of type FILE
W20260809 15:14:01.155416 21971 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:01.152785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37238 (local address 127.15.11.129:40207)
0809 15:14:01.153376 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.153384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.153409 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:14:01.153468 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.153474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.153674 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.153798 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.154822 (+ 1024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37238: BlockingRecv error: recv got EOF from 127.0.0.1:37238 (error 108)
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":301,"threads_started":1}
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:01.156963 15406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:01.158296 15406 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:14:01.173228 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:01.159843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37246 (local address 127.15.11.129:40207)
0809 15:14:01.160021 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.160026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.160222 (+ 196us) server_negotiation.cc:407] Connection header received
0809 15:14:01.160397 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.160402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.160467 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.160567 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.161574 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.162508 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.163716 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.164356 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.164508 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:01.167869 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:01.167879 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:01.167891 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:01.167920 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:01.170427 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.171121 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.171128 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.171130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.171249 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.171641 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.171647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.171650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.172002 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:01.172341 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:01.172699 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 15:14:01.172970 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 15:14:01.178848 21971 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:01.177045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37248 (local address 127.15.11.129:40207)
0809 15:14:01.177266 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.177272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.177517 (+ 245us) server_negotiation.cc:407] Connection header received
0809 15:14:01.177698 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.177703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.177765 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.177866 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.178608 (+ 742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37248: BlockingRecv error: recv got EOF from 127.0.0.1:37248 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 15:14:01.179447 15406 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5cc)
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:01.180815 15406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:01.182220 15406 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:14:01.196950 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:01.183550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.15.11.129:40207)
0809 15:14:01.183771 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.183778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.184142 (+ 364us) server_negotiation.cc:407] Connection header received
0809 15:14:01.184361 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.184367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.184436 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.184544 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.185548 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.186248 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.187576 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.188038 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.188217 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:01.191886 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:01.191893 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:01.191898 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:01.191937 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:01.194461 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.195096 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.195102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.195105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.195232 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.195623 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.195629 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.195632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.195876 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:01.196219 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:01.196479 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 15:14:01.196670 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 15:14:01.201426 21971 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:01.199694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37274 (local address 127.15.11.129:40207)
0809 15:14:01.199840 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.199844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.200282 (+ 438us) server_negotiation.cc:407] Connection header received
0809 15:14:01.200400 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.200403 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.200451 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.200522 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.201153 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37274: BlockingRecv error: recv got EOF from 127.0.0.1:37274 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 15:14:01.201687 15406 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786288392170773-15406-0/krb5kdc/krb5cc' of type FILE
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:01.203315 15406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:01.204828 15406 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:14:01.218541 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:01.206104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37278 (local address 127.15.11.129:40207)
0809 15:14:01.206296 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.206301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.206565 (+ 264us) server_negotiation.cc:407] Connection header received
0809 15:14:01.206733 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.206740 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.206797 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.206910 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.207911 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.208687 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.209900 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.210297 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.210433 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:01.213625 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:01.213630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:01.213633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:01.213661 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:01.216015 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.216710 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.216717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.216721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.216851 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.217227 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.217233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.217236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.217493 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:01.217834 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:01.218032 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 15:14:01.218239 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:14:01.220003 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 21529
I20260809 15:14:01.230996 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:01.220740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37284 (local address 127.15.11.129:40207)
0809 15:14:01.220958 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.220964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.221169 (+ 205us) server_negotiation.cc:407] Connection header received
0809 15:14:01.221276 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.221279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.221328 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.221405 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.222219 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.222818 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.223932 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.224422 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.224525 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:01.226285 (+ 1760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:01.226290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:01.226293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:01.226320 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:01.228833 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.229342 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.229345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.229347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.229469 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.229762 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.229765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.229767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.229988 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:01.230270 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:01.230483 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 15:14:01.230671 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:14:01.231544 21669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:01.313633 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 21529
I20260809 15:14:01.334873 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 21681
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:14:01.349972 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:01.335754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.15.11.130:39305)
0809 15:14:01.336381 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.336400 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.336425 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:14:01.336495 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.336502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.336708 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.336840 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.337985 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.338938 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.340081 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.340818 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.340913 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:01.344739 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:01.344751 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:01.344763 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:01.344795 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:01.347373 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.347938 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.347944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.347946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.348063 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.348610 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.348613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.348615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.348936 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:01.349145 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:01.349443 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 15:14:01.349683 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":338,"threads_started":1}
I20260809 15:14:01.351174 21811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:01.436241 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 21681
I20260809 15:14:01.457435 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 21822
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:14:01.473187 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:01.458574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58842 (local address 127.15.11.131:35441)
0809 15:14:01.459272 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:01.459279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:01.459306 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:14:01.459368 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:01.459374 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:01.459590 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:01.459718 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:01.460806 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.461781 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.463063 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:01.463614 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:01.463707 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:01.467719 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:01.467729 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:01.467743 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:01.467773 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:01.470625 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.471219 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.471226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.471228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.471341 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:01.471659 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:01.471662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:01.471664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:01.471986 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:01.472361 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:01.472613 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 15:14:01.472866 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":388,"threads_started":1}
I20260809 15:14:01.474488 21956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:01.559791 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 21822
I20260809 15:14:01.580204 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 21432
I20260809 15:14:01.581460 21497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:01.686215 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 21432
2026-08-09T15:14:01Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4025 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21992](info): set up 2 sockets
Aug 09 15:14:01 dist-test-slave-8w9v krb5kdc[21992](info): commencing operation
krb5kdc: starting...
W20260809 15:14:02.868839 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.008s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:14:03 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:14:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:14:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.190.keytab.
Entry for principal oryx/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.190.keytab.
I20260809 15:14:03.095541 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:45689
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.190.keytab
--principal=oryx/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:45689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:03.421934 22006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:03.422231 22006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:03.422394 22006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:03.429481 22006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:03.429574 22006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:03.429625 22006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:03.429733 22006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:03.429824 22006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:03.439424 22006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:45689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.190.keytab
--principal=oryx/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:45689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:03.440696 22006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:03.441956 22006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:03.453333 22013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:03.453284 22015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:03.453635 22012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:03.454275 22006 server_base.cc:1034] running on GCE node
I20260809 15:14:03.455018 22006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:03.456399 22006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:03.457603 22006 hybrid_clock.cc:648] HybridClock initialized: now 1786288443457563 us; error 36 us; skew 500 ppm
I20260809 15:14:03.457933 22006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:03 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:03.460942 22006 init.cc:377] Logged in from keytab as oryx/127.15.11.190@KRBTEST.COM (short username oryx)
I20260809 15:14:03.462456 22006 webserver.cc:460] Webserver started at http://127.15.11.190:45409/ using document root <none> and password file <none>
I20260809 15:14:03.463115 22006 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:03.463243 22006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:03.463578 22006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:03.466482 22006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "ca6f4feb52f942af9223513fffac1276"
format_stamp: "Formatted at 2026-08-09 15:14:03 on dist-test-slave-8w9v"
server_key: "f570f107d4dc3e3ca626c0a38347957e"
server_key_iv: "dbf2880b49a39ff86f5d48c8e14b453f"
server_key_version: "encryptionkey@0"
I20260809 15:14:03.467216 22006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "ca6f4feb52f942af9223513fffac1276"
format_stamp: "Formatted at 2026-08-09 15:14:03 on dist-test-slave-8w9v"
server_key: "f570f107d4dc3e3ca626c0a38347957e"
server_key_iv: "dbf2880b49a39ff86f5d48c8e14b453f"
server_key_version: "encryptionkey@0"
I20260809 15:14:03.472296 22006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 15:14:03.475426 22023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:03.476776 22006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:03.476954 22006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "ca6f4feb52f942af9223513fffac1276"
format_stamp: "Formatted at 2026-08-09 15:14:03 on dist-test-slave-8w9v"
server_key: "f570f107d4dc3e3ca626c0a38347957e"
server_key_iv: "dbf2880b49a39ff86f5d48c8e14b453f"
server_key_version: "encryptionkey@0"
I20260809 15:14:03.477138 22006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:03.496299 22006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:03.500188 22006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:03.500499 22006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:03.519228 22006 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:45689
I20260809 15:14:03.519269 22076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:45689 every 8 connection(s)
I20260809 15:14:03.520700 22006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:14:03.524451 22078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:03.526939 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 22006
I20260809 15:14:03.527165 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:14:03.527841 15406 external_mini_cluster.cc:1428] Setting key df5adb2dfef614168c0cea89a96dbf54
I20260809 15:14:03.536125 22078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: Bootstrap starting.
I20260809 15:14:03.539826 22078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:03.541196 22078 log.cc:824] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:14:03 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:03.544044 22078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: No bootstrap required, opened a new log
I20260809 15:14:03.550014 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:03.529794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.15.11.190:45689)
0809 15:14:03.530592 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:03.530603 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:03.530643 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:14:03.532296 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:03.532317 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:03.532672 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:03.532985 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:03.534708 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:03.535871 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:03.537606 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:03.538258 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:03.538386 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:03.542980 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:03.542987 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:03.542999 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:03.543034 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:03.546098 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:03.546785 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:03.546795 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:03.546806 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:03.546972 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:03.547341 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:03.547348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:03.547356 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:03.547834 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:03.548219 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:03.548672 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 15:14:03.549067 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":353,"threads_started":1}
I20260809 15:14:03.552043 22078 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } }
I20260809 15:14:03.552486 22078 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:03.552591 22078 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6f4feb52f942af9223513fffac1276, State: Initialized, Role: FOLLOWER
I20260809 15:14:03.553188 22078 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } }
I20260809 15:14:03.553416 22078 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:14:03.553521 22078 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:14:03.553639 22078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:03.555639 22078 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } }
I20260809 15:14:03.556072 22078 leader_election.cc:302] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca6f4feb52f942af9223513fffac1276; no voters:
I20260809 15:14:03.556793 22078 leader_election.cc:288] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:14:03.556912 22084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:03.557904 22084 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 1 LEADER]: Becoming Leader. State: Replica: ca6f4feb52f942af9223513fffac1276, State: Running, Role: LEADER
I20260809 15:14:03.558408 22084 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } }
I20260809 15:14:03.558894 22078 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:14:03.561204 22085 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } }
I20260809 15:14:03.561573 22085 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:03.562011 22086 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca6f4feb52f942af9223513fffac1276. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } }
I20260809 15:14:03.562320 22086 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:03.562409 22094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:14:03.568306 22094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:14:03.577328 22094 catalog_manager.cc:1347] Generated new cluster ID: 5fc86ddb90b045fbac39a635df58f458
I20260809 15:14:03.577468 22094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:14:03.593822 22094 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:14:03.595611 22094 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:14:03.602250 22094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: Generated new TSK 0
I20260809 15:14:03.603232 22094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab.
Entry for principal oryx/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab.
I20260809 15:14:03.669014 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45689
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab
--principal=oryx/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:03.997737 22107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:03.997995 22107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:03.998088 22107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:04.005263 22107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:04.005389 22107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:04.005563 22107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:14:04.015053 22107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab
--principal=oryx/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:04.016357 22107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:04.017716 22107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:04.030706 22113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:04.030682 22114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:04.030648 22116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:04.031845 22107 server_base.cc:1034] running on GCE node
I20260809 15:14:04.032536 22107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:04.033450 22107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:04.034644 22107 hybrid_clock.cc:648] HybridClock initialized: now 1786288444034602 us; error 29 us; skew 500 ppm
I20260809 15:14:04.034981 22107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:04 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288444, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:04.038101 22107 init.cc:377] Logged in from keytab as oryx/127.15.11.129@KRBTEST.COM (short username oryx)
I20260809 15:14:04.039412 22107 webserver.cc:460] Webserver started at http://127.15.11.129:40477/ using document root <none> and password file <none>
I20260809 15:14:04.040027 22107 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:04.040161 22107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:04.040448 22107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:04.043156 22107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "ee582d5158d04b8288b8207d1593112e"
format_stamp: "Formatted at 2026-08-09 15:14:04 on dist-test-slave-8w9v"
server_key: "55597c90eb90a57ced0d3c6a7ab9291e"
server_key_iv: "a174311098febd86060c63844768d12b"
server_key_version: "encryptionkey@0"
I20260809 15:14:04.043993 22107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "ee582d5158d04b8288b8207d1593112e"
format_stamp: "Formatted at 2026-08-09 15:14:04 on dist-test-slave-8w9v"
server_key: "55597c90eb90a57ced0d3c6a7ab9291e"
server_key_iv: "a174311098febd86060c63844768d12b"
server_key_version: "encryptionkey@0"
I20260809 15:14:04.048929 22107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 15:14:04.052398 22125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:04.053680 22107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:14:04.053853 22107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "ee582d5158d04b8288b8207d1593112e"
format_stamp: "Formatted at 2026-08-09 15:14:04 on dist-test-slave-8w9v"
server_key: "55597c90eb90a57ced0d3c6a7ab9291e"
server_key_iv: "a174311098febd86060c63844768d12b"
server_key_version: "encryptionkey@0"
I20260809 15:14:04.054087 22107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:04.086758 22107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:04.090672 22107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:04.090973 22107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:04.091925 22107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:04.093276 22107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:04.093376 22107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:04.093534 22107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:04.093606 22107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:04.126286 22107 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:46349
I20260809 15:14:04.126331 22251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:46349 every 8 connection(s)
I20260809 15:14:04.127596 22107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:14:04.131419 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 22107
I20260809 15:14:04.131631 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:14:04.132390 15406 external_mini_cluster.cc:1428] Setting key 7f7356bac1ba8f56c727164050930334
Aug 09 15:14:04 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288444, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:04.147884 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:04.130256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:44143 (local address 127.15.11.190:45689)
0809 15:14:04.130696 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:04.130700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:04.131859 (+ 1159us) server_negotiation.cc:407] Connection header received
0809 15:14:04.133662 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:04.133668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:04.133751 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:04.133880 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:04.135348 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:04.136102 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:04.137729 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:04.138132 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:04.138273 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:04.141840 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:04.141848 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:04.141855 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:04.141911 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:04.144771 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:04.145700 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:04.145704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:04.145706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:04.145831 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:04.146321 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:04.146324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:04.146326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:04.146568 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:04.146667 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:04.147465 (+ 798us) server_negotiation.cc:299] Negotiation successful
0809 15:14:04.147635 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":216,"threads_started":1}
I20260809 15:14:04.149451 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:04.130868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.129:44143)
0809 15:14:04.131690 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:04.131749 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:04.133414 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:04.134124 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:04.134132 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:04.134641 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:04.135182 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:04.135198 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:04.136317 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:04.136321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:04.137560 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:04.137573 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:04.138386 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:04.138395 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:04.138638 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:04.139272 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:04.139297 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:04.141619 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:04.144903 (+ 3284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:04.144914 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:04.144931 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:04.145573 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:04.145943 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:04.145950 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:04.145956 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:04.146214 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:04.146684 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:04.146694 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:04.147103 (+ 409us) client_negotiation.cc:769] Sending connection context
0809 15:14:04.147400 (+ 297us) client_negotiation.cc:240] Negotiation successful
0809 15:14:04.148001 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":313,"threads_started":1}
I20260809 15:14:04.151703 22252 heartbeater.cc:347] Connected to a master server at 127.15.11.190:45689
I20260809 15:14:04.152001 22252 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:04.152886 22252 heartbeater.cc:511] Master 127.15.11.190:45689 requested a full tablet report, sending...
I20260809 15:14:04.155306 22041 ts_manager.cc:194] Registered new tserver with Master: ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349)
I20260809 15:14:04.156869 22041 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:44143
WARNING: no policy specified for oryx/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab.
Entry for principal oryx/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab.
I20260809 15:14:04.186982 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45689
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab
--principal=oryx/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:04.537562 22258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:04.537860 22258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:04.538035 22258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:04.545295 22258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:04.545449 22258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:04.545629 22258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:14:04.555231 22258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab
--principal=oryx/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:04.556572 22258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:04.558036 22258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:04.571470 22266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:04.571470 22264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:04.571812 22263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:04.572492 22258 server_base.cc:1034] running on GCE node
I20260809 15:14:04.573094 22258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:04.573999 22258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:04.575196 22258 hybrid_clock.cc:648] HybridClock initialized: now 1786288444575148 us; error 32 us; skew 500 ppm
I20260809 15:14:04.575507 22258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:04 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288444, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:04.579048 22258 init.cc:377] Logged in from keytab as oryx/127.15.11.130@KRBTEST.COM (short username oryx)
I20260809 15:14:04.580610 22258 webserver.cc:460] Webserver started at http://127.15.11.130:36225/ using document root <none> and password file <none>
I20260809 15:14:04.581255 22258 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:04.581380 22258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:04.581740 22258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:04.584710 22258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "89ccdf564425469096c2e1f342c5d106"
format_stamp: "Formatted at 2026-08-09 15:14:04 on dist-test-slave-8w9v"
server_key: "b7129ef132f9dacb40e236d9ac9227de"
server_key_iv: "e30a1cf33a5d37af9a4e50246c980768"
server_key_version: "encryptionkey@0"
I20260809 15:14:04.585551 22258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "89ccdf564425469096c2e1f342c5d106"
format_stamp: "Formatted at 2026-08-09 15:14:04 on dist-test-slave-8w9v"
server_key: "b7129ef132f9dacb40e236d9ac9227de"
server_key_iv: "e30a1cf33a5d37af9a4e50246c980768"
server_key_version: "encryptionkey@0"
I20260809 15:14:04.590354 22258 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:14:04.593613 22274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:04.594895 22258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:14:04.595067 22258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "89ccdf564425469096c2e1f342c5d106"
format_stamp: "Formatted at 2026-08-09 15:14:04 on dist-test-slave-8w9v"
server_key: "b7129ef132f9dacb40e236d9ac9227de"
server_key_iv: "e30a1cf33a5d37af9a4e50246c980768"
server_key_version: "encryptionkey@0"
I20260809 15:14:04.595263 22258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:04.607522 22258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:04.611140 22258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:04.611418 22258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:04.612331 22258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:04.613570 22258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:04.613672 22258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:04.613811 22258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:04.613901 22258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:04.647217 22258 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:39013
I20260809 15:14:04.647261 22400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:39013 every 8 connection(s)
I20260809 15:14:04.648589 22258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:14:04.649523 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 22258
I20260809 15:14:04.649701 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:14:04.650293 15406 external_mini_cluster.cc:1428] Setting key 9d38b4db18d3f0e16ac81cf386b80df4
Aug 09 15:14:04 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288444, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:04.669771 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:04.651319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:49763 (local address 127.15.11.190:45689)
0809 15:14:04.651794 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:04.651800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:04.652958 (+ 1158us) server_negotiation.cc:407] Connection header received
0809 15:14:04.654763 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:04.654771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:04.654857 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:04.654979 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:04.656519 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:04.657604 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:04.659330 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:04.660071 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:04.660237 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:04.664259 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:04.664263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:04.664267 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:04.664297 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:04.666851 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:04.667518 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:04.667524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:04.667527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:04.667660 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:04.668077 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:04.668082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:04.668085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:04.668394 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:04.668552 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:04.669160 (+ 608us) server_negotiation.cc:299] Negotiation successful
0809 15:14:04.669363 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":240,"threads_started":1}
I20260809 15:14:04.670811 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:04.651941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:49763)
0809 15:14:04.652783 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:04.652836 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:04.654504 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:04.655181 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:04.655189 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:04.655689 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:04.656348 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:04.656375 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:04.657751 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:04.657755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:04.659170 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:04.659182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:04.660296 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:04.660301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:04.660497 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:04.661257 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:04.661289 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:04.663998 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:04.666993 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:04.667002 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:04.667016 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:04.667406 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:04.667774 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:04.667778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:04.667781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:04.667967 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:04.668527 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:04.668534 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:04.668833 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 15:14:04.669086 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 15:14:04.669600 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":386,"threads_started":1}
I20260809 15:14:04.672406 22401 heartbeater.cc:347] Connected to a master server at 127.15.11.190:45689
I20260809 15:14:04.672708 22401 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:04.673440 22401 heartbeater.cc:511] Master 127.15.11.190:45689 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.15.11.131@KRBTEST.COM; defaulting to no policy
I20260809 15:14:04.674953 22041 ts_manager.cc:194] Registered new tserver with Master: 89ccdf564425469096c2e1f342c5d106 (127.15.11.130:39013)
I20260809 15:14:04.675873 22041 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:49763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab.
Entry for principal oryx/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab.
I20260809 15:14:04.703419 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45689
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab
--principal=oryx/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:05.033157 22408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:05.033437 22408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:05.033535 22408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:05.040668 22408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:05.040761 22408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:05.040871 22408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:14:05.050426 22408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab
--principal=oryx/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:05.051761 22408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:05.053316 22408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:05.066965 22416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:05.067092 22413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:05.067022 22414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:05.067705 22408 server_base.cc:1034] running on GCE node
I20260809 15:14:05.068423 22408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:05.069416 22408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:05.070663 22408 hybrid_clock.cc:648] HybridClock initialized: now 1786288445070617 us; error 55 us; skew 500 ppm
I20260809 15:14:05.071005 22408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:05 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288445, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:05.079640 22408 init.cc:377] Logged in from keytab as oryx/127.15.11.131@KRBTEST.COM (short username oryx)
I20260809 15:14:05.081214 22408 webserver.cc:460] Webserver started at http://127.15.11.131:37209/ using document root <none> and password file <none>
I20260809 15:14:05.081918 22408 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:05.082064 22408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:05.082372 22408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:05.085402 22408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "9872f3dd69b54da6aaa37bd072bb6161"
format_stamp: "Formatted at 2026-08-09 15:14:05 on dist-test-slave-8w9v"
server_key: "f84717e6cbe18c26d90d0320c41f9958"
server_key_iv: "248a07b1d5a6e08154d3666ee36fb95f"
server_key_version: "encryptionkey@0"
I20260809 15:14:05.086220 22408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "9872f3dd69b54da6aaa37bd072bb6161"
format_stamp: "Formatted at 2026-08-09 15:14:05 on dist-test-slave-8w9v"
server_key: "f84717e6cbe18c26d90d0320c41f9958"
server_key_iv: "248a07b1d5a6e08154d3666ee36fb95f"
server_key_version: "encryptionkey@0"
I20260809 15:14:05.091420 22408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:14:05.094805 22425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:05.096335 22408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:05.096498 22408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "9872f3dd69b54da6aaa37bd072bb6161"
format_stamp: "Formatted at 2026-08-09 15:14:05 on dist-test-slave-8w9v"
server_key: "f84717e6cbe18c26d90d0320c41f9958"
server_key_iv: "248a07b1d5a6e08154d3666ee36fb95f"
server_key_version: "encryptionkey@0"
I20260809 15:14:05.096691 22408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:05.131968 22408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:05.136400 22408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:05.136708 22408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:05.137697 22408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:05.139093 22408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:05.139168 22408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:05.139354 22408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:05.139407 22408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:05.159891 22252 heartbeater.cc:503] Master 127.15.11.190:45689 was elected leader, sending a full tablet report...
I20260809 15:14:05.173659 22408 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:45485
I20260809 15:14:05.173780 22552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:45485 every 8 connection(s)
I20260809 15:14:05.175022 22408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:14:05.185179 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 22408
I20260809 15:14:05.185364 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:14:05.185817 15406 external_mini_cluster.cc:1428] Setting key d26d3dcce1cba60cf327290aee35b372
Aug 09 15:14:05 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288445, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:05.194537 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.177647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:47825 (local address 127.15.11.190:45689)
0809 15:14:05.178117 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:05.178139 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:05.179011 (+ 872us) server_negotiation.cc:407] Connection header received
0809 15:14:05.181025 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:05.181031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:05.181137 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:05.181294 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:05.182780 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.183549 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.185134 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.185631 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.185765 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.189580 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:05.189584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:05.189587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:05.189617 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:05.192049 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.192703 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.192707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.192709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.192822 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.193187 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.193190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.193192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.193423 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:05.193539 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:05.194132 (+ 593us) server_negotiation.cc:299] Negotiation successful
0809 15:14:05.194277 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":259,"threads_started":1}
I20260809 15:14:05.195423 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.178175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.131:47825)
0809 15:14:05.178861 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:05.178901 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:05.180765 (+ 1864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:05.181525 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:05.181533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:05.182061 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:05.182610 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.182627 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.183711 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.183715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.184990 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.185002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.185863 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.185870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.186097 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.186797 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:05.186834 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:05.189411 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:05.192224 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.192229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.192239 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.192607 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.192914 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.192917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.192919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.193089 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.193520 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:05.193525 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:05.193820 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 15:14:05.194036 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 15:14:05.194379 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":283,"threads_started":1}
I20260809 15:14:05.196738 22553 heartbeater.cc:347] Connected to a master server at 127.15.11.190:45689
I20260809 15:14:05.196995 22553 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:05.197696 22553 heartbeater.cc:511] Master 127.15.11.190:45689 requested a full tablet report, sending...
I20260809 15:14:05.199101 22041 ts_manager.cc:194] Registered new tserver with Master: 9872f3dd69b54da6aaa37bd072bb6161 (127.15.11.131:45485)
I20260809 15:14:05.200124 22041 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:47825
I20260809 15:14:05.201355 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.11.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.189.keytab.
Entry for principal oryx/127.15.11.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.189.keytab.
I20260809 15:14:05.256662 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.189:38029
--webserver_interface=127.15.11.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.189.keytab
--principal=oryx/127.15.11.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:45689,127.15.11.189:38029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:05.598412 22560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:05.598757 22560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:05.598903 22560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:05.606238 22560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:05.606350 22560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:05.606402 22560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:05.606462 22560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:05.606498 22560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:05.616547 22560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:45689,127.15.11.189:38029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.189.keytab
--principal=oryx/127.15.11.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.189:38029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.11.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
W20260809 15:14:05.616874 22560 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.11.190:45689,127.15.11.189:38029'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 15:14:05.617920 22560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:05.619231 22560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:05.631891 22565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:05.631858 22567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:05.632185 22569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:05.632758 22560 server_base.cc:1034] running on GCE node
I20260809 15:14:05.633574 22560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:05.634979 22560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:05.636255 22560 hybrid_clock.cc:648] HybridClock initialized: now 1786288445636187 us; error 56 us; skew 500 ppm
I20260809 15:14:05.636595 22560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:05 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288445, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:05.639884 22560 init.cc:377] Logged in from keytab as oryx/127.15.11.189@KRBTEST.COM (short username oryx)
I20260809 15:14:05.641434 22560 webserver.cc:460] Webserver started at http://127.15.11.189:37513/ using document root <none> and password file <none>
I20260809 15:14:05.642097 22560 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:05.642184 22560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:05.642457 22560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:05.645494 22560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data/instance:
uuid: "67ce804ea84f4b63999099f6206ede68"
format_stamp: "Formatted at 2026-08-09 15:14:05 on dist-test-slave-8w9v"
server_key: "c95e7bf55f2d9a21d4111dc14c0db216"
server_key_iv: "eccc80d50f250f5bf76ecc1161ad4b2c"
server_key_version: "encryptionkey@0"
I20260809 15:14:05.646533 22560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal/instance:
uuid: "67ce804ea84f4b63999099f6206ede68"
format_stamp: "Formatted at 2026-08-09 15:14:05 on dist-test-slave-8w9v"
server_key: "c95e7bf55f2d9a21d4111dc14c0db216"
server_key_iv: "eccc80d50f250f5bf76ecc1161ad4b2c"
server_key_version: "encryptionkey@0"
I20260809 15:14:05.652316 22560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:14:05.655676 22576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:05.657025 22560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:05.657243 22560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal
uuid: "67ce804ea84f4b63999099f6206ede68"
format_stamp: "Formatted at 2026-08-09 15:14:05 on dist-test-slave-8w9v"
server_key: "c95e7bf55f2d9a21d4111dc14c0db216"
server_key_iv: "eccc80d50f250f5bf76ecc1161ad4b2c"
server_key_version: "encryptionkey@0"
I20260809 15:14:05.657467 22560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:05.672950 22560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:05.677490 22560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:05.677788 22560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:05.678623 22401 heartbeater.cc:503] Master 127.15.11.190:45689 was elected leader, sending a full tablet report...
I20260809 15:14:05.697247 22560 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.189:38029
I20260809 15:14:05.697326 22632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.189:38029 every 8 connection(s)
I20260809 15:14:05.698705 22560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
I20260809 15:14:05.700467 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 22560
I20260809 15:14:05.703516 22633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:05.711922 22633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:14:05 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.189@KRBTEST.COM
Aug 09 15:14:05 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288445, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.189@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:05.724532 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.702788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.15.11.189:38029)
0809 15:14:05.703681 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:05.703692 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:05.703732 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:14:05.705394 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:05.705415 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:05.705759 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:05.706085 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:05.707919 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.709360 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.711154 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.711772 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.711886 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.717148 (+ 5262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:05.717159 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:05.717168 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:05.717204 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:05.720081 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.720959 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.720970 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.720979 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.721186 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.721629 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.721636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.721639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.722348 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:05.722660 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:05.723027 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 15:14:05.723440 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":381,"threads_started":1}
I20260809 15:14:05.730172 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.714954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35912 (local address 127.15.11.190:45689)
0809 15:14:05.716306 (+ 1352us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:05.716311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:05.716326 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:14:05.716375 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:05.716378 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:05.716448 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:05.716536 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:05.717905 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.718678 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.720086 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.720817 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.720958 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.724654 (+ 3696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:05.724662 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:05.724666 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:05.724708 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:05.727435 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.728115 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.728152 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.728155 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.728297 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.728673 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.728679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.728682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.729010 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:05.729164 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:05.729605 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 15:14:05.729830 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1240,"thread_start_us":1090,"threads_started":1}
I20260809 15:14:05.730239 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.714845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:35912)
0809 15:14:05.715578 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:05.715625 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:05.715793 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:05.716645 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:05.716653 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:05.717399 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:05.717736 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.717747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.718827 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.718832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.719942 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.719954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.721022 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.721033 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.721214 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.721831 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:05.721917 (+ 86us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:05.724443 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:05.727567 (+ 3124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.727577 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.727583 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.727972 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.728408 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.728413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.728416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.728561 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.729164 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:05.729172 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:05.729326 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 15:14:05.729508 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 15:14:05.729807 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"spinlock_wait_cycles":14592,"thread_start_us":318,"threads_started":1}
I20260809 15:14:05.731876 22633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:14:05 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288445, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.189@KRBTEST.COM for oryx/127.15.11.189@KRBTEST.COM
I20260809 15:14:05.742889 22637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.732649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.0.0.1:56616)
0809 15:14:05.732830 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:05.732849 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:05.732944 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:05.733185 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:05.733188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:05.733466 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:05.733590 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.733597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.734469 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.734473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.735319 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.735329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.735949 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.735953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.736082 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.736633 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:05.736649 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:05.738442 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:05.741014 (+ 2572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.741018 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.741020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.741368 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.741696 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.741698 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.741701 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.741818 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.742249 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:05.742252 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:05.742370 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 15:14:05.742492 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 15:14:05.742675 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 15:14:05.743148 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.732585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56616 (local address 127.15.11.189:38029)
0809 15:14:05.732722 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:05.732726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:05.732918 (+ 192us) server_negotiation.cc:407] Connection header received
0809 15:14:05.733025 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:05.733028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:05.733077 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:05.733150 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:05.733701 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.734346 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.735437 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.735830 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.735917 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.738601 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:05.738606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:05.738608 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:05.738634 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:05.740896 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.741465 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.741468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.741470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.741593 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.741905 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.741908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.741910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.742143 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:05.742246 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:05.742569 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 15:14:05.742809 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":43}
I20260809 15:14:05.749788 22633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Bootstrap starting.
I20260809 15:14:05.753774 22633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:05.754823 22633 log.cc:824] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:05.757453 22633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: No bootstrap required, opened a new log
I20260809 15:14:05.764559 22633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } }
I20260809 15:14:05.764972 22633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:05.765080 22633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67ce804ea84f4b63999099f6206ede68, State: Initialized, Role: FOLLOWER
I20260809 15:14:05.765625 22633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } }
I20260809 15:14:05.766433 22640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } } }
I20260809 15:14:05.766713 22640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:14:05.767352 22633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:14:05.776290 22652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:14:05.776434 22652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:14:05.779287 22560 master_runner.cc:428] Detected that this master 67ce804ea84f4b63999099f6206ede68 is joining an existing cluster
I20260809 15:14:05.779389 22560 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.11.189:38029
I20260809 15:14:05.782640 22640 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:05.782951 22640 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } }
I20260809 15:14:05.784372 22640 leader_election.cc:288] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689)
I20260809 15:14:05.786010 22051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67ce804ea84f4b63999099f6206ede68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca6f4feb52f942af9223513fffac1276" is_pre_election: true
I20260809 15:14:05.787532 22578 leader_election.cc:302] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 67ce804ea84f4b63999099f6206ede68; no voters: ca6f4feb52f942af9223513fffac1276
I20260809 15:14:05.788499 22640 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
WARNING: no policy specified for oryx/127.15.11.188@KRBTEST.COM; defaulting to no policy
I20260809 15:14:05.809687 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.784936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.15.11.190:45689)
0809 15:14:05.785163 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:05.785172 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:05.790332 (+ 5160us) server_negotiation.cc:407] Connection header received
0809 15:14:05.790635 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:05.790642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:05.790732 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:05.790790 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:05.791991 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.793153 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.795555 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.796370 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.796483 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.799458 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:05.799467 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:05.799474 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:05.799518 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:05.803742 (+ 4224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.806957 (+ 3215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.806963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.806967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.807139 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.807721 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.807727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.807731 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.808192 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:05.808329 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:05.809068 (+ 739us) server_negotiation.cc:299] Negotiation successful
0809 15:14:05.809344 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:14:05.809886 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.789665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.15.11.189:38029)
0809 15:14:05.789870 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:05.789878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:05.793053 (+ 3175us) server_negotiation.cc:407] Connection header received
0809 15:14:05.793250 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:05.793261 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:05.793330 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:05.793411 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:05.794890 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.796027 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.798600 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.799380 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.799479 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.802614 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:05.802622 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:05.802627 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:05.802661 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:05.806646 (+ 3985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.807550 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.807556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.807559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.807709 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.808238 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.808243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.808246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.808602 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:05.808780 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:05.809287 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 15:14:05.809516 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:14:05.810796 22658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.783738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:35926)
0809 15:14:05.790217 (+ 6479us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:05.790244 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:05.790469 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:05.790965 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:05.790968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:05.791497 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:05.791837 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.791846 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.793322 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.793331 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.795357 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.795377 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.796792 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.796800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.797046 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.797800 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:05.797828 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:05.799241 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:05.803931 (+ 4690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.803938 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.803943 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.806802 (+ 2859us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.807311 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.807317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.807323 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.807571 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.808448 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:05.808455 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:05.808714 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 15:14:05.809985 (+ 1271us) client_negotiation.cc:240] Negotiation successful
0809 15:14:05.810460 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6325,"mutex_wait_us":1,"thread_start_us":6168,"threads_started":1}
I20260809 15:14:05.811179 22660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.792266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.0.0.1:56626)
0809 15:14:05.792963 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:05.792981 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:05.793132 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:05.793620 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:05.793625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:05.794355 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:05.794720 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.794733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.796264 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.796269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.798382 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.798398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.799920 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.799927 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.800117 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.800996 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:05.801019 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:05.802399 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:05.806797 (+ 4398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.806802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.806807 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.807417 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.807847 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.807852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.807855 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.808072 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.808860 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:05.808864 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:05.809015 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 15:14:05.810063 (+ 1048us) client_negotiation.cc:240] Negotiation successful
0809 15:14:05.810678 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":294,"spinlock_wait_cycles":35584,"thread_start_us":5087,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.188@KRBTEST.COM" created.
Aug 09 15:14:05 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.190@KRBTEST.COM for oryx/127.15.11.189@KRBTEST.COM
I20260809 15:14:05.840255 22636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.820084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56642 (local address 127.15.11.189:38029)
0809 15:14:05.820284 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:05.820292 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:05.822943 (+ 2651us) server_negotiation.cc:407] Connection header received
0809 15:14:05.823291 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:05.823297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:05.823401 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:05.823511 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:05.824720 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.825983 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.827926 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.828795 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.828948 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.833161 (+ 4213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:05.833168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:05.833173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:05.833220 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:05.837020 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.837883 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.837888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.837892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.838073 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.838603 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.838609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.838612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.838964 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:05.839108 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:05.839592 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 15:14:05.839837 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:14:05.840272 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.819730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.0.0.1:56642)
0809 15:14:05.822817 (+ 3087us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:05.822853 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:05.823075 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:05.823571 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:05.823579 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:05.824324 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:05.824559 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.824568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.826146 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.826153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.827766 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.827784 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.828978 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.828985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.829190 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.829896 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:05.829931 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:05.832952 (+ 3021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:05.837168 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.837183 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.837195 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.837733 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.838210 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.838213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.838215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.838457 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.839107 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:05.839113 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:05.839271 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 15:14:05.839450 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 15:14:05.839788 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2864,"thread_start_us":2730,"threads_started":1}
I20260809 15:14:05.842630 22041 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.11.189:38029: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "67ce804ea84f4b63999099f6206ede68"
member_type: NON_VOTER
last_known_addr {
host: "127.15.11.189"
port: 38029
}
attrs {
promote: true
}
}
dest_uuid: "ca6f4feb52f942af9223513fffac1276"
cas_config_opid_index: -1
I20260809 15:14:05.843674 22041 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.188.keytab.
Entry for principal oryx/127.15.11.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.188.keytab.
I20260809 15:14:05.846570 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.188:33459
--webserver_interface=127.15.11.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.188.keytab
--principal=oryx/127.15.11.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:45689,127.15.11.189:38029,127.15.11.188:33459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:14:05.851892 22607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:05.851433 22665 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 67ce804ea84f4b63999099f6206ede68 (127.15.11.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } } }
I20260809 15:14:05.855157 22607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Refusing update from remote peer ca6f4feb52f942af9223513fffac1276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 15:14:05.856022 22664 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:05.864027 22665 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.11.189:38029
I20260809 15:14:05.866173 22560 master.cc:537] Master@127.15.11.189:38029 shutting down...
I20260809 15:14:05.867775 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } } }
I20260809 15:14:05.868206 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:05.913800 22607 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 67ce804ea84f4b63999099f6206ede68 (127.15.11.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } } }
I20260809 15:14:05.918992 22675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } } }
I20260809 15:14:05.919176 22640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca6f4feb52f942af9223513fffac1276. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } } }
I20260809 15:14:05.919525 22640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:14:05.919771 22675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:14:05.919554 22671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } } }
I20260809 15:14:05.919941 22671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:14:05.922611 22560 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 LEARNER]: Raft consensus shutting down.
I20260809 15:14:05.923134 22560 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 LEARNER]: Raft consensus is shut down!
I20260809 15:14:05.923292 22560 tablet_replica.cc:330] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: stopping tablet replica
I20260809 15:14:05.932240 22560 master.cc:552] Master@127.15.11.189:38029 shutdown complete.
I20260809 15:14:05.932390 22560 master_runner.cc:305] Clearing existing system tablet
I20260809 15:14:05.932893 22560 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:05.933071 22560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:05.936336 22560 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:14:05.938225 22677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:05.938602 22560 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:05.939062 22560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 15:14:05.939203 22560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal
uuid: "67ce804ea84f4b63999099f6206ede68"
format_stamp: "Formatted at 2026-08-09 15:14:05 on dist-test-slave-8w9v"
server_key: "c95e7bf55f2d9a21d4111dc14c0db216"
server_key_iv: "eccc80d50f250f5bf76ecc1161ad4b2c"
server_key_version: "encryptionkey@0"
I20260809 15:14:05.943919 22560 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:05.951200 22560 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 15:14:05.951457 22560 log.cc:1196] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 15:14:05.952065 22560 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Deleting consensus metadata
I20260809 15:14:05.952421 22560 master_runner.cc:315] Copying system tablet from 127.15.11.190:45689
I20260809 15:14:05.955152 22560 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: tablet copy: Beginning tablet copy session from remote peer at address 127.15.11.190:45689
I20260809 15:14:05.972339 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.956165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.15.11.190:45689)
0809 15:14:05.956502 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:05.956512 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:05.956837 (+ 325us) server_negotiation.cc:407] Connection header received
0809 15:14:05.957048 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:05.957054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:05.957181 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:05.957299 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:05.958542 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.959789 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.961891 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.962536 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.962674 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.965151 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:05.965160 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:05.965167 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:05.965208 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:05.969062 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.969940 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.969945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.969949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.970117 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:05.970593 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:05.970599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:05.970602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:05.970997 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:05.971163 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:05.971665 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 15:14:05.971910 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260809 15:14:05.972410 22686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:05.956104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:35940)
0809 15:14:05.956699 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:05.956726 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:05.956899 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:05.957507 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:05.957512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:05.958074 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:05.958371 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.958385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.960029 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.960036 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.961704 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:05.961721 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:05.962715 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:05.962721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:05.962911 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:05.963662 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:05.963686 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:05.964944 (+ 1258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:05.969203 (+ 4259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.969212 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.969217 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.969793 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.970262 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:05.970267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:05.970271 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:05.970446 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:05.971145 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:05.971150 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:05.971350 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 15:14:05.971512 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 15:14:05.971861 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"spinlock_wait_cycles":1792,"thread_start_us":380,"threads_started":1}
I20260809 15:14:05.973587 22061 tablet_copy_service.cc:140] P ca6f4feb52f942af9223513fffac1276: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 67ce804ea84f4b63999099f6206ede68 ({username='oryx', principal='oryx/127.15.11.189@KRBTEST.COM'} at 127.0.0.1:35940)
I20260809 15:14:05.973877 22061 tablet_copy_service.cc:161] P ca6f4feb52f942af9223513fffac1276: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 67ce804ea84f4b63999099f6206ede68 at {username='oryx', principal='oryx/127.15.11.189@KRBTEST.COM'} at 127.0.0.1:35940: session id = 67ce804ea84f4b63999099f6206ede68-00000000000000000000000000000000
I20260809 15:14:05.978904 22061 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:14:05.981309 22560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 15:14:05.988287 22560 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: tablet copy: Starting download of 0 data blocks...
I20260809 15:14:05.988615 22560 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: tablet copy: Starting download of 1 WAL segments...
I20260809 15:14:05.991380 22560 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:14:05.995965 22061 tablet_copy_service.cc:342] P ca6f4feb52f942af9223513fffac1276: Request end of tablet copy session 67ce804ea84f4b63999099f6206ede68-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.11.189@KRBTEST.COM'} at 127.0.0.1:35940
I20260809 15:14:05.996197 22061 tablet_copy_service.cc:433] P ca6f4feb52f942af9223513fffac1276: ending tablet copy session 67ce804ea84f4b63999099f6206ede68-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 67ce804ea84f4b63999099f6206ede68
I20260809 15:14:06.008270 22560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:06.010897 22687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:06.020525 22688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:06.020947 22690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:06.023978 22560 server_base.cc:1034] running on GCE node
I20260809 15:14:06.024403 22560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:06.025081 22560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:06.026364 22560 hybrid_clock.cc:648] HybridClock initialized: now 1786288446026299 us; error 34 us; skew 500 ppm
I20260809 15:14:06.026597 22560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:06 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288446, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:06.028677 22560 init.cc:377] Logged in from keytab as oryx/127.15.11.189@KRBTEST.COM (short username oryx)
I20260809 15:14:06.037184 22560 webserver.cc:460] Webserver started at http://127.15.11.189:41645/ using document root <none> and password file <none>
I20260809 15:14:06.037585 22560 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:06.037694 22560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:06.040424 22560 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:14:06.042194 22697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:06.042654 22560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 15:14:06.042819 22560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal
uuid: "67ce804ea84f4b63999099f6206ede68"
format_stamp: "Formatted at 2026-08-09 15:14:05 on dist-test-slave-8w9v"
server_key: "c95e7bf55f2d9a21d4111dc14c0db216"
server_key_iv: "eccc80d50f250f5bf76ecc1161ad4b2c"
server_key_version: "encryptionkey@0"
I20260809 15:14:06.043010 22560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:06.052856 22560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:06.056592 22560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:06.094555 22560 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.189:38029
I20260809 15:14:06.094597 22764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.189:38029 every 8 connection(s)
I20260809 15:14:06.095796 22560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
I20260809 15:14:06.101099 22765 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:14:06.105495 22765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Bootstrap starting.
I20260809 15:14:06.124254 22765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:14:06.124827 22765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Bootstrap complete.
I20260809 15:14:06.126104 22765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } }
I20260809 15:14:06.126410 22765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 67ce804ea84f4b63999099f6206ede68, State: Initialized, Role: LEARNER
I20260809 15:14:06.126708 22765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } }
I20260809 15:14:06.129900 22768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } } }
I20260809 15:14:06.130347 22765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:14:06.130427 22768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:14:06.138669 22779 catalog_manager.cc:1259] Loaded cluster ID: 5fc86ddb90b045fbac39a635df58f458
I20260809 15:14:06.138892 22779 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: loading cluster ID for follower catalog manager: success
I20260809 15:14:06.142472 22779 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: acquiring CA information for follower catalog manager: success
I20260809 15:14:06.145025 22779 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:14:06.202806 22553 heartbeater.cc:503] Master 127.15.11.190:45689 was elected leader, sending a full tablet report...
W20260809 15:14:06.265098 22667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:06.265377 22667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:06.265547 22667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:06.272706 22667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:06.272827 22667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:06.272907 22667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:06.273005 22667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:06.273082 22667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:06.276779 22665 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } } }
I20260809 15:14:06.277072 22665 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:06.283777 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.277182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.0.0.1:56656)
0809 15:14:06.277444 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:06.277463 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:06.277588 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:06.278384 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:06.278389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:06.278424 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:06.278625 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.278636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.279878 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.279885 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.281828 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.281843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.282974 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.282980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.283235 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.283291 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 15:14:06.283381 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 15:14:06.283528 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260809 15:14:06.283793 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.277469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.15.11.189:38029)
0809 15:14:06.278035 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.278040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.278056 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:14:06.278127 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.278131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.278197 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.278309 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:06.278775 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.279716 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.281974 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.282818 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.282908 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.283075 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.283442 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.283534 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":330,"threads_started":1}
I20260809 15:14:06.284807 22768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca6f4feb52f942af9223513fffac1276. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: NON_VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: true } } }
I20260809 15:14:06.285065 22768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:14:06.285207 22667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:45689,127.15.11.189:38029,127.15.11.188:33459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.188.keytab
--principal=oryx/127.15.11.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.188:33459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.11.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:06.285616 22665 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: attempting to promote NON_VOTER 67ce804ea84f4b63999099f6206ede68 to VOTER
I20260809 15:14:06.286360 22665 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } }
I20260809 15:14:06.286636 22667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:06.287998 22732 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 LEARNER]: Refusing update from remote peer ca6f4feb52f942af9223513fffac1276: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 15:14:06.288097 22667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:14:06.288583 22664 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:14:06.291765 22665 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 67ce804ea84f4b63999099f6206ede68 (127.15.11.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } }
I20260809 15:14:06.292485 22732 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 67ce804ea84f4b63999099f6206ede68 (127.15.11.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } }
I20260809 15:14:06.294812 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } }
I20260809 15:14:06.295136 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:06.294889 22666 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } }
I20260809 15:14:06.295291 22666 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:06.297834 22768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } }
I20260809 15:14:06.298041 22768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 15:14:06.303342 22786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:06.303436 22789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:06.303440 22787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:06.304325 22667 server_base.cc:1034] running on GCE node
I20260809 15:14:06.305027 22667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:06.306393 22667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:06.307585 22667 hybrid_clock.cc:648] HybridClock initialized: now 1786288446307530 us; error 47 us; skew 500 ppm
I20260809 15:14:06.307940 22667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:06 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288446, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:06.311154 22667 init.cc:377] Logged in from keytab as oryx/127.15.11.188@KRBTEST.COM (short username oryx)
I20260809 15:14:06.312788 22667 webserver.cc:460] Webserver started at http://127.15.11.188:43971/ using document root <none> and password file <none>
I20260809 15:14:06.313465 22667 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:06.313603 22667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:06.313898 22667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:06.316891 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data/instance:
uuid: "fa97ebad4c6d41db9ca14369f747596d"
format_stamp: "Formatted at 2026-08-09 15:14:06 on dist-test-slave-8w9v"
server_key: "c3f6a64d7dfb06a176fe226da3c6ae2d"
server_key_iv: "4a14396ff4e477297439eef8dd8f0246"
server_key_version: "encryptionkey@0"
I20260809 15:14:06.317744 22667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal/instance:
uuid: "fa97ebad4c6d41db9ca14369f747596d"
format_stamp: "Formatted at 2026-08-09 15:14:06 on dist-test-slave-8w9v"
server_key: "c3f6a64d7dfb06a176fe226da3c6ae2d"
server_key_iv: "4a14396ff4e477297439eef8dd8f0246"
server_key_version: "encryptionkey@0"
I20260809 15:14:06.322825 22667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:14:06.326293 22797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:06.327639 22667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:14:06.327822 22667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal
uuid: "fa97ebad4c6d41db9ca14369f747596d"
format_stamp: "Formatted at 2026-08-09 15:14:06 on dist-test-slave-8w9v"
server_key: "c3f6a64d7dfb06a176fe226da3c6ae2d"
server_key_iv: "4a14396ff4e477297439eef8dd8f0246"
server_key_version: "encryptionkey@0"
I20260809 15:14:06.328016 22667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:06.357945 22667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:06.362265 22667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:06.362568 22667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:06.381417 22667 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.188:33459
I20260809 15:14:06.381433 22850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.188:33459 every 8 connection(s)
I20260809 15:14:06.382822 22667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
I20260809 15:14:06.384624 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 22667
I20260809 15:14:06.387526 22851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:06.394286 22851 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:14:06 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.188@KRBTEST.COM
Aug 09 15:14:06 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288446, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.188@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:06.408656 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.386822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60568 (local address 127.15.11.188:33459)
0809 15:14:06.387656 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.387673 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.387725 (+ 52us) server_negotiation.cc:407] Connection header received
0809 15:14:06.389370 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.389401 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.389770 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.390169 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.391621 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.393015 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.394219 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.394896 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.395079 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.400511 (+ 5432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.400521 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.400532 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.400575 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.404304 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.405056 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.405061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.405067 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.405227 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.405620 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.405623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.405625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.406150 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.406356 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.406902 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.407282 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":369,"threads_started":1}
I20260809 15:14:06.412719 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.396952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.15.11.190:45689)
0809 15:14:06.397120 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.397124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.397748 (+ 624us) server_negotiation.cc:407] Connection header received
0809 15:14:06.397953 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.397963 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.398027 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.398126 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.399625 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.400547 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.402281 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.402987 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.403133 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.406722 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.406729 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.406733 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.406771 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.410083 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.410845 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.410852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.410855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.411015 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.411479 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.411483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.411485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.411737 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.411855 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.412321 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.412510 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:14:06.412735 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.396916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:35942)
0809 15:14:06.397609 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:06.397642 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:06.397820 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:06.398185 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:06.398193 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:06.399056 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:06.399415 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.399430 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.400745 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.400752 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.402119 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.402146 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.403173 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.403181 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.403303 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.403883 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:06.403902 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:06.406517 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:06.410213 (+ 3696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.410224 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.410230 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.410711 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.411198 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.411203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.411206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.411372 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.411852 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:06.411857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:06.412000 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 15:14:06.412249 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 15:14:06.412469 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":356,"threads_started":1}
I20260809 15:14:06.414002 22851 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:14:06 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288446, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.188@KRBTEST.COM for oryx/127.15.11.189@KRBTEST.COM
I20260809 15:14:06.426260 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.414733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.0.0.1:56660)
0809 15:14:06.414947 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:06.414965 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:06.415087 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:06.415396 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:06.415401 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:06.415722 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:06.415838 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.415845 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.416937 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.416944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.417847 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.417877 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.418611 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.418616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.418782 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.419225 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:06.419240 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:06.421041 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:06.423859 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.423873 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.423877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.424305 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.424689 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.424694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.424698 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.424836 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.425347 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:06.425351 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:06.425490 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 15:14:06.425625 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 15:14:06.425929 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 15:14:06.426296 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.414736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.15.11.189:38029)
0809 15:14:06.414940 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.414946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.415034 (+ 88us) server_negotiation.cc:407] Connection header received
0809 15:14:06.415199 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.415206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.415276 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.415395 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.415994 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.416804 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.418004 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.418457 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.418584 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.421252 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.421257 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.421260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.421293 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.423700 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.424423 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.424429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.424432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.424569 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.424952 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.424957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.424962 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.425225 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.425390 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.425718 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.425966 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:14:06.427356 22851 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:14:06 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288446, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.188@KRBTEST.COM for oryx/127.15.11.188@KRBTEST.COM
I20260809 15:14:06.438866 22855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.427935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33459 (local address 127.0.0.1:60582)
0809 15:14:06.428114 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:06.428190 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:06.428271 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:06.428514 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:06.428517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:06.428831 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:06.428973 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.428979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.429843 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.429851 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.430775 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.430787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.431576 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.431581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.431724 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.432217 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:06.432245 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:06.434114 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:06.436956 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.436960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.436962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.437303 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.437668 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.437671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.437673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.437819 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.438224 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:06.438227 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:06.438349 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 15:14:06.438452 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 15:14:06.438608 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 15:14:06.439042 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.428022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.15.11.188:33459)
0809 15:14:06.428212 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.428215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.428233 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:14:06.428335 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.428338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.428389 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.428475 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.429080 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.429721 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.430881 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.431382 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.431558 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.434306 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.434310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.434313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.434340 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.436835 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.437408 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.437411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.437413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.437534 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.437904 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.437907 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.437909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.438136 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.438236 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.438532 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.438720 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":60}
I20260809 15:14:06.443730 22851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: Bootstrap starting.
I20260809 15:14:06.446869 22851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:06.447906 22851 log.cc:824] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:06.450527 22851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: No bootstrap required, opened a new log
I20260809 15:14:06.457437 22851 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } }
I20260809 15:14:06.457749 22851 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:06.457867 22851 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa97ebad4c6d41db9ca14369f747596d, State: Initialized, Role: FOLLOWER
I20260809 15:14:06.458593 22851 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } }
I20260809 15:14:06.459493 22857 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } } }
I20260809 15:14:06.459875 22857 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:14:06.460564 22851 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:14:06.469915 22869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:14:06.470075 22869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:14:06.473820 22667 master_runner.cc:428] Detected that this master fa97ebad4c6d41db9ca14369f747596d is joining an existing cluster
I20260809 15:14:06.473912 22667 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.11.188:33459
I20260809 15:14:06.496951 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.478136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35952 (local address 127.15.11.190:45689)
0809 15:14:06.478336 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.478344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.479137 (+ 793us) server_negotiation.cc:407] Connection header received
0809 15:14:06.479208 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.479213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.479294 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.479413 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.481425 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.482623 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.485228 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.485882 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.486014 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.488701 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.488710 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.488716 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.488758 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.492312 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.493123 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.493128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.493132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.493289 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.493752 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.493757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.493760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.494091 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.494229 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.496359 (+ 2130us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.496597 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:14:06.497524 22875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.478179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:35952)
0809 15:14:06.478909 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:06.478931 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:06.479088 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:06.479746 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:06.479753 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:06.480702 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:06.481206 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.481224 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.482741 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.482748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.485034 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.485054 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.486242 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.486256 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.486470 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.487193 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:06.487218 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:06.488494 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:06.492507 (+ 4013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.492511 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.492516 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.492972 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.493433 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.493438 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.493441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.493614 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.494289 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:06.494294 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:06.494539 (+ 245us) client_negotiation.cc:769] Sending connection context
0809 15:14:06.494688 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 15:14:06.495014 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":395,"threads_started":1}
I20260809 15:14:06.508358 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.479642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.15.11.188:33459)
0809 15:14:06.479839 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.479846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.490043 (+ 10197us) server_negotiation.cc:407] Connection header received
0809 15:14:06.490209 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.490217 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.490304 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.490420 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.492110 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.493478 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.495679 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.496485 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.496593 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.500311 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.500320 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.500327 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.500366 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.504312 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.506144 (+ 1832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.506151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.506154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.506304 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.506710 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.506715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.506718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.507048 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.507199 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.507739 (+ 540us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.507960 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:14:06.514497 22876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.479089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33459 (local address 127.0.0.1:60596)
0809 15:14:06.489934 (+ 10845us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:06.489950 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:06.490093 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:06.490604 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:06.490611 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:06.491503 (+ 892us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:06.491932 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.491948 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.493709 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.493716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.495540 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.495550 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.496876 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.496883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.497085 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.497726 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:06.497769 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:06.500094 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:06.505020 (+ 4926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.505026 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.505030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.506014 (+ 984us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.506438 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.506442 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.506445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.506602 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.507278 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:06.507282 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:06.507457 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 15:14:06.513678 (+ 6221us) client_negotiation.cc:240] Negotiation successful
0809 15:14:06.514023 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10712,"mutex_wait_us":1,"thread_start_us":10545,"threads_started":1}
I20260809 15:14:06.515183 22783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.478082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56670 (local address 127.15.11.189:38029)
0809 15:14:06.478264 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.478270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.495225 (+ 16955us) server_negotiation.cc:407] Connection header received
0809 15:14:06.495704 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.495710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.495793 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.495858 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.497656 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.498738 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.501137 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.501702 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.501852 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.504700 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.504705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.504708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.504741 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.509021 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.512032 (+ 3011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.512040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.512046 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.512313 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.512882 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.512889 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.512894 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.513342 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.513537 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.514358 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.514691 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:14:06.516435 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.479409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.0.0.1:56670)
0809 15:14:06.495119 (+ 15710us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:06.495138 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:06.495544 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:06.496093 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:06.496097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:06.496795 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:06.497451 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.497467 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.498966 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.498970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.500964 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.500981 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.501909 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.501916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.502128 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.503015 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:06.503044 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:06.504500 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:06.511548 (+ 7048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.511552 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.511554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.511882 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.512501 (+ 619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.512509 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.512515 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.512734 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.513641 (+ 907us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:06.513648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:06.513979 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 15:14:06.515504 (+ 1525us) client_negotiation.cc:240] Negotiation successful
0809 15:14:06.515879 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15587,"mutex_wait_us":28,"thread_start_us":15830,"threads_started":1}
Aug 09 15:14:06 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.190@KRBTEST.COM for oryx/127.15.11.188@KRBTEST.COM
I20260809 15:14:06.527173 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 22107
I20260809 15:14:06.529513 22661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.505220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33459 (local address 127.0.0.1:60606)
0809 15:14:06.505432 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:06.505451 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:06.505558 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:06.509293 (+ 3735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:06.509298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:06.509786 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:06.510011 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.510024 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.511673 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.511679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.513057 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.513073 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.516348 (+ 3275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.516355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.516628 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.517557 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:06.517579 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:06.522397 (+ 4818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:06.526695 (+ 4298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.526700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.526704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.527191 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.527620 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.527625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.527628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.527791 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.528473 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:06.528478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:06.528655 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 15:14:06.528802 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 15:14:06.529123 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 15:14:06.530503 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.505628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60606 (local address 127.15.11.188:33459)
0809 15:14:06.508903 (+ 3275us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.508908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.508925 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:14:06.509008 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.509013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.509116 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.510165 (+ 1049us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.510334 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.511506 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.513635 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.515885 (+ 2250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.517041 (+ 1156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.522599 (+ 5558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.522605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.522610 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.522644 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.526538 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.527342 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.527348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.527355 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.527489 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.527942 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.527951 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.527954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.528307 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.529760 (+ 1453us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.529887 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.530094 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1165,"server-negotiator.queue_time_us":3180,"thread_start_us":20364,"threads_started":1}
I20260809 15:14:06.533414 22041 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.11.188:33459: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d"
member_type: NON_VOTER
last_known_addr {
host: "127.15.11.188"
port: 33459
}
attrs {
promote: true
}
}
dest_uuid: "ca6f4feb52f942af9223513fffac1276"
cas_config_opid_index: 6
I20260809 15:14:06.534233 22041 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } }
I20260809 15:14:06.536911 22732 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Refusing update from remote peer ca6f4feb52f942af9223513fffac1276: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 15:14:06.537544 22666 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:14:06.540511 22666 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER fa97ebad4c6d41db9ca14369f747596d (127.15.11.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } } }
I20260809 15:14:06.544090 22732 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER fa97ebad4c6d41db9ca14369f747596d (127.15.11.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } } }
Aug 09 15:14:06 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.129@KRBTEST.COM
I20260809 15:14:06.545356 22825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:06.547931 22768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } } }
I20260809 15:14:06.548237 22768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:14:06.549808 22825 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Refusing update from remote peer ca6f4feb52f942af9223513fffac1276: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 15:14:06.550570 22665 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:06.551190 22666 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.11.188:33459
I20260809 15:14:06.551733 22667 master.cc:537] Master@127.15.11.188:33459 shutting down...
I20260809 15:14:06.555814 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.528513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58026 (local address 127.15.11.129:46349)
0809 15:14:06.533032 (+ 4519us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.533041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.533073 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:14:06.533162 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.533170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.533543 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.535539 (+ 1996us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.535792 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.537447 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.539399 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.540332 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.546060 (+ 5728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.546365 (+ 305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.546377 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.546404 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.546443 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.550280 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.551168 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.551177 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.551181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.551328 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.553054 (+ 1726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.553060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.553063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.553495 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.554974 (+ 1479us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.555136 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.555411 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4282,"thread_start_us":21859,"threads_started":1}
I20260809 15:14:06.557520 22247 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 15:14:06.558501 22025 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 15:14:06.569840 22025 consensus_peers.cc:487] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 -> Peer fa97ebad4c6d41db9ca14369f747596d (127.15.11.188:33459): Couldn't send request to peer fa97ebad4c6d41db9ca14369f747596d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:14:06.570744 22665 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } } }
I20260809 15:14:06.570976 22665 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:06.571384 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } } }
I20260809 15:14:06.571578 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:06.583089 22667 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:06.583495 22667 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:06.583745 22667 tablet_replica.cc:330] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: stopping tablet replica
I20260809 15:14:06.590147 22667 master.cc:552] Master@127.15.11.188:33459 shutdown complete.
I20260809 15:14:06.590250 22667 master_runner.cc:305] Clearing existing system tablet
I20260809 15:14:06.590596 22667 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:06.590730 22667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:06.593078 22667 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:14:06.594599 22884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:06.594971 22667 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:06.595240 22667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 15:14:06.595346 22667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal
uuid: "fa97ebad4c6d41db9ca14369f747596d"
format_stamp: "Formatted at 2026-08-09 15:14:06 on dist-test-slave-8w9v"
server_key: "c3f6a64d7dfb06a176fe226da3c6ae2d"
server_key_iv: "4a14396ff4e477297439eef8dd8f0246"
server_key_version: "encryptionkey@0"
I20260809 15:14:06.599396 22667 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:06.604346 22667 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 15:14:06.604517 22667 log.cc:1196] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 15:14:06.604964 22667 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: Deleting consensus metadata
I20260809 15:14:06.605232 22667 master_runner.cc:315] Copying system tablet from 127.15.11.190:45689
I20260809 15:14:06.609719 22667 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: tablet copy: Beginning tablet copy session from remote peer at address 127.15.11.190:45689
I20260809 15:14:06.622819 22638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.610844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.15.11.190:45689)
0809 15:14:06.611088 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:06.611096 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:06.611123 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:14:06.611429 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:06.611435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:06.611529 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:06.611642 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:06.612567 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.613305 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.614568 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.614990 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.615135 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.617002 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:06.617008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:06.617013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:06.617055 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:06.619895 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.620740 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.620745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.620752 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.620908 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:06.621338 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:06.621342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:06.621344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:06.621611 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:06.621768 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:06.622204 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 15:14:06.622448 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260809 15:14:06.622872 22893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:06.610494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:35958)
0809 15:14:06.611031 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:06.611058 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:06.611213 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:06.611671 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:06.611681 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:06.612191 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:06.612403 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.612414 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.613464 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.613471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.614411 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:06.614423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:06.615145 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:06.615151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:06.615332 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:06.615978 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:06.616007 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:06.616782 (+ 775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:06.620029 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.620035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.620040 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.620597 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.621047 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:06.621050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:06.621052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:06.621210 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:06.621738 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:06.621742 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:06.621910 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 15:14:06.622078 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 15:14:06.622403 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":643,"threads_started":1}
I20260809 15:14:06.623723 22061 tablet_copy_service.cc:140] P ca6f4feb52f942af9223513fffac1276: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fa97ebad4c6d41db9ca14369f747596d ({username='oryx', principal='oryx/127.15.11.188@KRBTEST.COM'} at 127.0.0.1:35958)
I20260809 15:14:06.623898 22061 tablet_copy_service.cc:161] P ca6f4feb52f942af9223513fffac1276: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fa97ebad4c6d41db9ca14369f747596d at {username='oryx', principal='oryx/127.15.11.188@KRBTEST.COM'} at 127.0.0.1:35958: session id = fa97ebad4c6d41db9ca14369f747596d-00000000000000000000000000000000
I20260809 15:14:06.626505 22061 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:14:06.628080 22667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 15:14:06.633347 22667 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: tablet copy: Starting download of 0 data blocks...
I20260809 15:14:06.633709 22667 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: tablet copy: Starting download of 1 WAL segments...
I20260809 15:14:06.635968 22667 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:14:06.639284 22061 tablet_copy_service.cc:342] P ca6f4feb52f942af9223513fffac1276: Request end of tablet copy session fa97ebad4c6d41db9ca14369f747596d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.11.188@KRBTEST.COM'} at 127.0.0.1:35958
I20260809 15:14:06.639413 22061 tablet_copy_service.cc:433] P ca6f4feb52f942af9223513fffac1276: ending tablet copy session fa97ebad4c6d41db9ca14369f747596d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fa97ebad4c6d41db9ca14369f747596d
I20260809 15:14:06.651926 22667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:06.654707 22894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:06.655083 22895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:06.662360 22667 server_base.cc:1034] running on GCE node
W20260809 15:14:06.662681 22898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:06.663187 22667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:06.663882 22667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:06.665136 22667 hybrid_clock.cc:648] HybridClock initialized: now 1786288446665061 us; error 60 us; skew 500 ppm
I20260809 15:14:06.665452 22667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:06 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288446, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:06.667766 22667 init.cc:377] Logged in from keytab as oryx/127.15.11.188@KRBTEST.COM (short username oryx)
I20260809 15:14:06.673091 22667 webserver.cc:460] Webserver started at http://127.15.11.188:39929/ using document root <none> and password file <none>
I20260809 15:14:06.673524 22667 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:06.673688 22667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:06.676927 22667 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:14:06.678987 22909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:06.679471 22667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 15:14:06.679636 22667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal
uuid: "fa97ebad4c6d41db9ca14369f747596d"
format_stamp: "Formatted at 2026-08-09 15:14:06 on dist-test-slave-8w9v"
server_key: "c3f6a64d7dfb06a176fe226da3c6ae2d"
server_key_iv: "4a14396ff4e477297439eef8dd8f0246"
server_key_version: "encryptionkey@0"
I20260809 15:14:06.679908 22667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:06.701035 22667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:06.705476 22667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:06.714116 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 22107
I20260809 15:14:06.754258 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:46349
--local_ip_for_outbound_sockets=127.15.11.129
--tserver_master_addrs=127.15.11.190:45689
--webserver_port=40477
--webserver_interface=127.15.11.129
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab
--principal=oryx/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.11.190:45689,127.15.11.189:38029
--tserver_master_addrs=127.15.11.190:45689,127.15.11.189:38029,127.15.11.188:33459 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:14:06.755247 22667 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.188:33459
I20260809 15:14:06.755275 22961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.188:33459 every 8 connection(s)
I20260809 15:14:06.756228 22667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
I20260809 15:14:06.763366 22962 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:14:06.768414 22962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: Bootstrap starting.
I20260809 15:14:06.786872 22962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:14:06.787530 22962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: Bootstrap complete.
I20260809 15:14:06.788805 22962 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } }
I20260809 15:14:06.789062 22962 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fa97ebad4c6d41db9ca14369f747596d, State: Initialized, Role: LEARNER
I20260809 15:14:06.789311 22962 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } }
I20260809 15:14:06.791373 22966 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } } }
I20260809 15:14:06.791714 22962 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:14:06.791813 22966 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: This master's current role is: LEARNER
I20260809 15:14:06.802239 22977 catalog_manager.cc:1259] Loaded cluster ID: 5fc86ddb90b045fbac39a635df58f458
I20260809 15:14:06.802363 22977 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: loading cluster ID for follower catalog manager: success
I20260809 15:14:06.804869 22977 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: acquiring CA information for follower catalog manager: success
I20260809 15:14:06.806784 22977 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:14:07.068830 22978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.061536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33459 (local address 127.0.0.1:60618)
0809 15:14:07.062098 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:07.062125 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:07.062280 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:07.062767 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:07.062774 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:07.062805 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:07.063064 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.063076 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.065137 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.065142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.066326 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.066338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.067979 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.067983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.068106 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.068195 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 15:14:07.068407 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 15:14:07.068542 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":291,"threads_started":1}
I20260809 15:14:07.069020 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.061834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60618 (local address 127.15.11.188:33459)
0809 15:14:07.062435 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:07.062439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:07.062454 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:14:07.062514 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:07.062517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:07.062585 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:07.062716 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:07.063308 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.064974 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.066510 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.067826 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.067978 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.068270 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:07.068463 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 15:14:07.068632 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":300,"threads_started":1}
I20260809 15:14:07.070034 22937 raft_consensus.cc:1203] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.7] Dedup: 1.7->[]
I20260809 15:14:07.070626 22966 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca6f4feb52f942af9223513fffac1276. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } } }
I20260809 15:14:07.070988 22966 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: This master's current role is: LEARNER
W20260809 15:14:07.154879 22964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:07.155156 22964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:07.155246 22964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:07.162323 22964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:07.162426 22964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:07.162540 22964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:14:07.171612 22964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab
--principal=oryx/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:46349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=40477
--tserver_master_addrs=127.15.11.190:45689,127.15.11.189:38029,127.15.11.188:33459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:07.172843 22964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:07.174497 22964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:07.188656 22988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:07.188704 22986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:07.189216 22985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:07.193550 22964 server_base.cc:1034] running on GCE node
I20260809 15:14:07.194166 22964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:07.195091 22964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:07.196313 22964 hybrid_clock.cc:648] HybridClock initialized: now 1786288447196278 us; error 50 us; skew 500 ppm
I20260809 15:14:07.196636 22964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288447, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:07.199954 22964 init.cc:377] Logged in from keytab as oryx/127.15.11.129@KRBTEST.COM (short username oryx)
I20260809 15:14:07.201538 22964 webserver.cc:460] Webserver started at http://127.15.11.129:40477/ using document root <none> and password file <none>
I20260809 15:14:07.202258 22964 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:07.202421 22964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:07.208035 22964 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:14:07.210726 22995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:07.212044 22964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:14:07.212242 22964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "ee582d5158d04b8288b8207d1593112e"
format_stamp: "Formatted at 2026-08-09 15:14:04 on dist-test-slave-8w9v"
server_key: "55597c90eb90a57ced0d3c6a7ab9291e"
server_key_iv: "a174311098febd86060c63844768d12b"
server_key_version: "encryptionkey@0"
I20260809 15:14:07.213238 22964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:07.246994 22964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:07.250953 22964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:07.251253 22964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:07.252267 22964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:07.253628 22964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:07.253752 22964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:07.253887 22964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:07.253978 22964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:07.288009 22964 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:46349
I20260809 15:14:07.288103 23120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:46349 every 8 connection(s)
I20260809 15:14:07.289515 22964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:14:07.294021 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 22964
I20260809 15:14:07.294477 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 22258
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.130@KRBTEST.COM
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288447, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.188@KRBTEST.COM
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288447, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.189@KRBTEST.COM
I20260809 15:14:07.323992 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.295593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33590 (local address 127.15.11.130:39013)
0809 15:14:07.297062 (+ 1469us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:07.297073 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:07.297112 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:14:07.297207 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:07.297218 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:07.297530 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:07.300669 (+ 3139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:07.301028 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.303084 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.306139 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.307270 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.311929 (+ 4659us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.315026 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:07.315042 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:07.315060 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:07.315100 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:07.320502 (+ 5402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.321475 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.321488 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.321493 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.321661 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.322131 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.322136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.322139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.322605 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:07.322921 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:07.323283 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 15:14:07.323564 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1240,"thread_start_us":1001,"threads_started":1}
I20260809 15:14:07.325558 22396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:07.326933 22979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.292414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:59645 (local address 127.15.11.188:33459)
0809 15:14:07.292619 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:07.292626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:07.300904 (+ 8278us) server_negotiation.cc:407] Connection header received
0809 15:14:07.304541 (+ 3637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:07.304550 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:07.304653 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:07.304759 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:07.307684 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.309082 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.312097 (+ 3015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.313059 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.313193 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.318868 (+ 5675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:07.318876 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:07.318880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:07.318922 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:07.322785 (+ 3863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.323686 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.323693 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.323697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.323864 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.324577 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.324582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.324585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.324941 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:07.325079 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:07.326123 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0809 15:14:07.326579 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288447, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:07.330330 23126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.293232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.15.11.129:53625)
0809 15:14:07.304301 (+ 11069us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:07.304319 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:07.310617 (+ 6298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:07.311415 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:07.311422 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:07.312195 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:07.312510 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.312523 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.314237 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.314244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.315933 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.315955 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.317331 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.317337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.317733 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.318431 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:07.318457 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:07.321617 (+ 3160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:07.326205 (+ 4588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.326210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.326214 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.326709 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.327671 (+ 962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.327678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.327681 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.327816 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.328626 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:07.328660 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:07.328811 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 15:14:07.328947 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 15:14:07.329203 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10897,"mutex_wait_us":380,"thread_start_us":6659,"threads_started":1}
I20260809 15:14:07.330372 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.293232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33459 (local address 127.15.11.129:59645)
0809 15:14:07.300747 (+ 7515us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:07.300793 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:07.304210 (+ 3417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:07.305434 (+ 1224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:07.305450 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:07.306433 (+ 983us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:07.307405 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.307435 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.309343 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.309350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.311918 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.311932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.313636 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.313645 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.314028 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.314959 (+ 931us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:07.315005 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:07.318586 (+ 3581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:07.322949 (+ 4363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.322957 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.322973 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.323557 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.324164 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.324170 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.324174 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.324428 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.325200 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:07.325208 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:07.325644 (+ 436us) client_negotiation.cc:769] Sending connection context
0809 15:14:07.327137 (+ 1493us) client_negotiation.cc:240] Negotiation successful
0809 15:14:07.328787 (+ 1650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7201,"thread_start_us":10561,"threads_started":1}
I20260809 15:14:07.331720 23125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.292391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:53625 (local address 127.15.11.189:38029)
0809 15:14:07.303655 (+ 11264us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:07.303659 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:07.308567 (+ 4908us) server_negotiation.cc:407] Connection header received
0809 15:14:07.310813 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:07.310819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:07.310912 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:07.311025 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:07.312731 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.314046 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.316159 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.316875 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.316980 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.321816 (+ 4836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:07.321822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:07.321825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:07.321859 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:07.325508 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.326845 (+ 1337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.326851 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.326856 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.327000 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.327994 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.327999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.328003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.328381 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:07.328550 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:07.331093 (+ 2543us) server_negotiation.cc:299] Negotiation successful
0809 15:14:07.331330 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11148,"thread_start_us":11021,"threads_started":1}
I20260809 15:14:07.332464 23121 heartbeater.cc:347] Connected to a master server at 127.15.11.188:33459
I20260809 15:14:07.332566 23123 heartbeater.cc:347] Connected to a master server at 127.15.11.189:38029
I20260809 15:14:07.332832 23121 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:07.332830 23123 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:07.333891 23121 heartbeater.cc:511] Master 127.15.11.188:33459 requested a full tablet report, sending...
I20260809 15:14:07.333993 23123 heartbeater.cc:511] Master 127.15.11.189:38029 requested a full tablet report, sending...
I20260809 15:14:07.336443 22722 ts_manager.cc:194] Registered new tserver with Master: ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349)
I20260809 15:14:07.336444 22927 ts_manager.cc:194] Registered new tserver with Master: ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349)
I20260809 15:14:07.338829 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.304480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.129:56061)
0809 15:14:07.311431 (+ 6951us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:07.311449 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:07.311569 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:07.317186 (+ 5617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:07.317190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:07.317971 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:07.318358 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.318388 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.319765 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.319772 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.321381 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.321397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.325734 (+ 4337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.325742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.325946 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.327285 (+ 1339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:07.327306 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:07.331582 (+ 4276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:07.336227 (+ 4645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.336232 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.336235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.336789 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.337260 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.337267 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.337272 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.337482 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.338156 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:07.338159 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:07.338283 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 15:14:07.338398 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 15:14:07.338580 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6825,"thread_start_us":7182,"threads_started":1}
I20260809 15:14:07.339318 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.304584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:56061 (local address 127.15.11.190:45689)
0809 15:14:07.316821 (+ 12237us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:07.316827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:07.316857 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:14:07.316966 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:07.316971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:07.317044 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:07.317165 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:07.318572 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.319645 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.323711 (+ 4066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.324367 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.324499 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.332046 (+ 7547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:07.332054 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:07.332060 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:07.332109 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:07.335921 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.336953 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.336960 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.336964 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.337127 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.337624 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.337631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.337636 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.338021 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:07.338172 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:07.338555 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 15:14:07.338852 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12076,"thread_start_us":12736,"threads_started":1}
I20260809 15:14:07.340095 23122 heartbeater.cc:347] Connected to a master server at 127.15.11.190:45689
I20260809 15:14:07.340240 23122 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:07.340647 23122 heartbeater.cc:511] Master 127.15.11.190:45689 requested a full tablet report, sending...
I20260809 15:14:07.341619 22041 ts_manager.cc:194] Re-registered known tserver with Master: ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349)
I20260809 15:14:07.342370 22041 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:56061
I20260809 15:14:07.410926 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 22258
I20260809 15:14:07.432375 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:39013
--local_ip_for_outbound_sockets=127.15.11.130
--tserver_master_addrs=127.15.11.190:45689
--webserver_port=36225
--webserver_interface=127.15.11.130
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab
--principal=oryx/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.11.190:45689,127.15.11.189:38029
--tserver_master_addrs=127.15.11.190:45689,127.15.11.189:38029,127.15.11.188:33459 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:14:07.452137 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: NON_VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: true } } }
I20260809 15:14:07.452605 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:07.454241 22664 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: attempting to promote NON_VOTER fa97ebad4c6d41db9ca14369f747596d to VOTER
I20260809 15:14:07.454829 22664 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } }
I20260809 15:14:07.456722 22937 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 LEARNER]: Refusing update from remote peer ca6f4feb52f942af9223513fffac1276: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 15:14:07.456754 22732 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Refusing update from remote peer ca6f4feb52f942af9223513fffac1276: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 15:14:07.457288 23134 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:14:07.457779 23135 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:14:07.460367 23134 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fa97ebad4c6d41db9ca14369f747596d (127.15.11.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:07.461217 22732 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fa97ebad4c6d41db9ca14369f747596d (127.15.11.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:07.464695 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:07.464972 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:07.465658 22664 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:07.465873 22664 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:07.466872 23138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:07.467293 23138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:14:07.469313 23134 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:07.469599 23134 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:07.470021 22937 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, fa97ebad4c6d41db9ca14369f747596d (127.15.11.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:07.473215 22966 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ca6f4feb52f942af9223513fffac1276" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:07.473469 22966 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: This master's current role is: FOLLOWER
W20260809 15:14:07.779295 23133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:07.779605 23133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:07.779772 23133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:07.787310 23133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:07.787459 23133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:07.787648 23133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:14:07.797140 23133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab
--principal=oryx/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:39013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=36225
--tserver_master_addrs=127.15.11.190:45689,127.15.11.189:38029,127.15.11.188:33459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:07.798422 23133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:07.799752 23133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:07.813817 23143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:07.813731 23144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:07.813769 23146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:07.814648 23133 server_base.cc:1034] running on GCE node
I20260809 15:14:07.815217 23133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:07.816164 23133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:07.817379 23133 hybrid_clock.cc:648] HybridClock initialized: now 1786288447817348 us; error 63 us; skew 500 ppm
I20260809 15:14:07.817673 23133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288447, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:07.821197 23133 init.cc:377] Logged in from keytab as oryx/127.15.11.130@KRBTEST.COM (short username oryx)
I20260809 15:14:07.822609 23133 webserver.cc:460] Webserver started at http://127.15.11.130:36225/ using document root <none> and password file <none>
I20260809 15:14:07.823221 23133 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:07.823344 23133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:07.829099 23133 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:14:07.831957 23153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:07.833523 23133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:14:07.833683 23133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "89ccdf564425469096c2e1f342c5d106"
format_stamp: "Formatted at 2026-08-09 15:14:04 on dist-test-slave-8w9v"
server_key: "b7129ef132f9dacb40e236d9ac9227de"
server_key_iv: "e30a1cf33a5d37af9a4e50246c980768"
server_key_version: "encryptionkey@0"
I20260809 15:14:07.834714 23133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:07.850096 23133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:07.854005 23133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:07.854315 23133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:07.855326 23133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:07.856771 23133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:07.856885 23133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:07.857010 23133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:07.857110 23133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:07.891938 23133 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:39013
I20260809 15:14:07.892035 23280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:39013 every 8 connection(s)
I20260809 15:14:07.893574 23133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:14:07.896240 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 23133
I20260809 15:14:07.896625 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 22408
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.131@KRBTEST.COM
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288447, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.189@KRBTEST.COM
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288447, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.188@KRBTEST.COM
Aug 09 15:14:07 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288447, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:07.944180 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.897510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:48825 (local address 127.15.11.189:38029)
0809 15:14:07.917145 (+ 19635us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:07.917149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:07.919328 (+ 2179us) server_negotiation.cc:407] Connection header received
0809 15:14:07.923354 (+ 4026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:07.923362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:07.923480 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:07.923570 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:07.925928 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.927076 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.930189 (+ 3113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.930822 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.930931 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.936016 (+ 5085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:07.936021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:07.936024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:07.936057 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:07.939812 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.941268 (+ 1456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.941273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.941277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.941424 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.941919 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.941924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.941927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.942317 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:07.942491 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:07.943556 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0809 15:14:07.943773 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19536,"thread_start_us":19337,"threads_started":1}
I20260809 15:14:07.944199 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.896671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:38691 (local address 127.15.11.188:33459)
0809 15:14:07.917422 (+ 20751us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:07.917426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:07.921880 (+ 4454us) server_negotiation.cc:407] Connection header received
0809 15:14:07.922708 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:07.922714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:07.922804 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:07.922908 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:07.925764 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.926916 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.930277 (+ 3361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.931042 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.931137 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.936564 (+ 5427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:07.936572 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:07.936579 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:07.936622 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:07.940193 (+ 3571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.941344 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.941352 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.941357 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.941513 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.942022 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.942030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.942036 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.942361 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:07.942493 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:07.943552 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0809 15:14:07.943774 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20652,"thread_start_us":20565,"threads_started":1}
I20260809 15:14:07.944304 23290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.898184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.15.11.131:45485)
0809 15:14:07.916691 (+ 18507us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:07.916701 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:07.918370 (+ 1669us) server_negotiation.cc:407] Connection header received
0809 15:14:07.918608 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:07.918615 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:07.918835 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:07.919030 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:07.921240 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.922755 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.924756 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.925625 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.925764 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.932847 (+ 7083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:07.932883 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:07.932899 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:07.932940 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:07.938547 (+ 5607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.939402 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.939412 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.939418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.939567 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.939959 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.939964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.939968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.940442 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:07.940767 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:07.943022 (+ 2255us) server_negotiation.cc:299] Negotiation successful
0809 15:14:07.943400 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18294,"thread_start_us":18120,"threads_started":1}
I20260809 15:14:07.945739 22548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:07.946702 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.897391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.15.11.130:48825)
0809 15:14:07.919159 (+ 21768us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:07.919217 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:07.922292 (+ 3075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:07.924984 (+ 2692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:07.924988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:07.925444 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:07.925752 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.925764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.927306 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.927311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.930024 (+ 2713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.930040 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.931263 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.931270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.931779 (+ 509us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.932721 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:07.932764 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:07.935773 (+ 3009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:07.940349 (+ 4576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.940353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.940356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.941141 (+ 785us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.941570 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.941576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.941582 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.941794 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.942578 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:07.942586 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:07.943074 (+ 488us) client_negotiation.cc:769] Sending connection context
0809 15:14:07.944576 (+ 1502us) client_negotiation.cc:240] Negotiation successful
0809 15:14:07.945127 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21463,"mutex_wait_us":554,"spinlock_wait_cycles":29440,"thread_start_us":25218,"threads_started":1}
I20260809 15:14:07.946738 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.898043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33459 (local address 127.15.11.130:38691)
0809 15:14:07.921699 (+ 23656us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:07.921718 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:07.922305 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:07.923401 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:07.923412 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:07.924533 (+ 1121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:07.925546 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.925572 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.927135 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.927143 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.930052 (+ 2909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.930076 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.931515 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.931574 (+ 59us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.931798 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.932887 (+ 1089us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:07.932911 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:07.936348 (+ 3437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:07.940328 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.940336 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.940351 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.941189 (+ 838us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.941648 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.941655 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.941659 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.941838 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.942578 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:07.942586 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:07.943074 (+ 488us) client_negotiation.cc:769] Sending connection context
0809 15:14:07.945050 (+ 1976us) client_negotiation.cc:240] Negotiation successful
0809 15:14:07.945269 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23517,"mutex_wait_us":495,"spinlock_wait_cycles":36992,"thread_start_us":20712,"threads_started":1}
I20260809 15:14:07.948570 23281 heartbeater.cc:347] Connected to a master server at 127.15.11.188:33459
I20260809 15:14:07.949003 23281 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:07.949067 23283 heartbeater.cc:347] Connected to a master server at 127.15.11.189:38029
I20260809 15:14:07.949224 23283 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:07.949023 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.919965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:48443)
0809 15:14:07.923026 (+ 3061us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:07.923045 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:07.923151 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:07.923843 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:07.923847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:07.924494 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:07.925545 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.925572 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.927306 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.927311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.930037 (+ 2726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:07.930047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.931263 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.931270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:07.931744 (+ 474us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.932720 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:07.932764 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:07.940212 (+ 7448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:07.944418 (+ 4206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.944423 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.944427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.944921 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.945483 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:07.945488 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:07.945491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:07.945676 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:07.948050 (+ 2374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:07.948055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:07.948229 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 15:14:07.948386 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 15:14:07.948641 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2933,"mutex_wait_us":786,"spinlock_wait_cycles":30208,"thread_start_us":1390,"threads_started":1}
I20260809 15:14:07.950152 23281 heartbeater.cc:511] Master 127.15.11.188:33459 requested a full tablet report, sending...
I20260809 15:14:07.950306 23283 heartbeater.cc:511] Master 127.15.11.189:38029 requested a full tablet report, sending...
I20260809 15:14:07.951279 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:07.920112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:48443 (local address 127.15.11.190:45689)
0809 15:14:07.923520 (+ 3408us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:07.923528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:07.923549 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:14:07.923610 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:07.923614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:07.923706 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:07.923816 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:07.926014 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.927089 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.930195 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:07.930851 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:07.930972 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:07.940418 (+ 9446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:07.940426 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:07.940431 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:07.940477 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:07.944046 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.945071 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.945077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.945080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.945256 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:07.946709 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:07.946718 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:07.946724 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:07.947186 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:07.947353 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:07.950416 (+ 3063us) server_negotiation.cc:299] Negotiation successful
0809 15:14:07.950733 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3259,"thread_start_us":3812,"threads_started":1}
I20260809 15:14:07.952373 22722 ts_manager.cc:194] Registered new tserver with Master: 89ccdf564425469096c2e1f342c5d106 (127.15.11.130:39013)
I20260809 15:14:07.952476 23282 heartbeater.cc:347] Connected to a master server at 127.15.11.190:45689
I20260809 15:14:07.952616 23282 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:07.952903 22927 ts_manager.cc:194] Registered new tserver with Master: 89ccdf564425469096c2e1f342c5d106 (127.15.11.130:39013)
I20260809 15:14:07.953162 23282 heartbeater.cc:511] Master 127.15.11.190:45689 requested a full tablet report, sending...
I20260809 15:14:07.954308 22041 ts_manager.cc:194] Re-registered known tserver with Master: 89ccdf564425469096c2e1f342c5d106 (127.15.11.130:39013)
I20260809 15:14:07.955580 22041 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:48443
I20260809 15:14:08.036454 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 22408
I20260809 15:14:08.057715 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:45485
--local_ip_for_outbound_sockets=127.15.11.131
--tserver_master_addrs=127.15.11.190:45689
--webserver_port=37209
--webserver_interface=127.15.11.131
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab
--principal=oryx/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.11.190:45689,127.15.11.189:38029
--tserver_master_addrs=127.15.11.190:45689,127.15.11.189:38029,127.15.11.188:33459 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:14:08.344851 23122 heartbeater.cc:503] Master 127.15.11.190:45689 was elected leader, sending a full tablet report...
W20260809 15:14:08.395324 23295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:08.395596 23295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:08.395772 23295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:08.403040 23295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:08.403165 23295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:08.403353 23295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:14:08.413053 23295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab
--principal=oryx/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:45485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=37209
--tserver_master_addrs=127.15.11.190:45689,127.15.11.189:38029,127.15.11.188:33459
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:08.414382 23295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:08.415723 23295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:08.429419 23301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:08.429457 23304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:08.429800 23300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:08.430356 23295 server_base.cc:1034] running on GCE node
I20260809 15:14:08.430931 23295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:08.431999 23295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:08.433261 23295 hybrid_clock.cc:648] HybridClock initialized: now 1786288448433190 us; error 53 us; skew 500 ppm
I20260809 15:14:08.433580 23295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:08 dist-test-slave-8w9v krb5kdc[21992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288448, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:08.436976 23295 init.cc:377] Logged in from keytab as oryx/127.15.11.131@KRBTEST.COM (short username oryx)
I20260809 15:14:08.438597 23295 webserver.cc:460] Webserver started at http://127.15.11.131:37209/ using document root <none> and password file <none>
I20260809 15:14:08.439330 23295 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:08.439484 23295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:08.444830 23295 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:14:08.447319 23311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:08.448597 23295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:08.448773 23295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "9872f3dd69b54da6aaa37bd072bb6161"
format_stamp: "Formatted at 2026-08-09 15:14:05 on dist-test-slave-8w9v"
server_key: "f84717e6cbe18c26d90d0320c41f9958"
server_key_iv: "248a07b1d5a6e08154d3666ee36fb95f"
server_key_version: "encryptionkey@0"
I20260809 15:14:08.449702 23295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:08.491663 23295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:08.495792 23295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:08.496204 23295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:08.497287 23295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:08.498632 23295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:08.498755 23295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:08.498879 23295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:08.498991 23295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:08.538599 23295 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:45485
I20260809 15:14:08.538683 23434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:45485 every 8 connection(s)
I20260809 15:14:08.540249 23295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:14:08.541427 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 23295
Aug 09 15:14:08 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288448, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.189@KRBTEST.COM
Aug 09 15:14:08 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288448, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
Aug 09 15:14:08 dist-test-slave-8w9v krb5kdc[21992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288448, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.188@KRBTEST.COM
I20260809 15:14:08.597914 23444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.544909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:50659 (local address 127.15.11.190:45689)
0809 15:14:08.565392 (+ 20483us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.565401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.565424 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:14:08.565495 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.565503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.565609 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.568742 (+ 3133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:08.568971 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.570413 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.577220 (+ 6807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.577959 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.581538 (+ 3579us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.585434 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:08.585443 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:08.585449 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:08.585491 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:08.591184 (+ 5693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.592493 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.592502 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.592508 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.592698 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.595400 (+ 2702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.595408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.595414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.595876 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:08.596896 (+ 1020us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.597054 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.597605 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19433,"thread_start_us":19333,"threads_started":1}
I20260809 15:14:08.598268 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.543759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.131:50659)
0809 15:14:08.563096 (+ 19337us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:08.563118 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:08.563530 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:08.566080 (+ 2550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:08.566095 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:08.567315 (+ 1220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:08.568486 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.568539 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.574565 (+ 6026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.574573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.577020 (+ 2447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.577035 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.578252 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.578262 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.578616 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.579677 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:08.579707 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:08.585179 (+ 5472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:08.591324 (+ 6145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:08.591335 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:08.591355 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:08.592022 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:08.592819 (+ 797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:08.592828 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:08.592833 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:08.593119 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:08.596031 (+ 2912us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:08.596038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:08.596267 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 15:14:08.596448 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 15:14:08.596801 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19150,"mutex_wait_us":180,"spinlock_wait_cycles":59008,"thread_start_us":7920,"threads_started":1}
I20260809 15:14:08.598740 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.543759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.15.11.131:37083)
0809 15:14:08.552305 (+ 8546us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:08.552349 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:08.557522 (+ 5173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:08.570681 (+ 13159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:08.570688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:08.571273 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:08.571664 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.571679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.575285 (+ 3606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.575291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.577106 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.577132 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.578613 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.578622 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.578898 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.579681 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:08.579729 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:08.583847 (+ 4118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:08.592912 (+ 9065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:08.592919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:08.592926 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:08.593536 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:08.594059 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:08.594066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:08.594072 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:08.594247 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:08.595021 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:08.595032 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:08.595566 (+ 534us) client_negotiation.cc:769] Sending connection context
0809 15:14:08.595915 (+ 349us) client_negotiation.cc:240] Negotiation successful
0809 15:14:08.596621 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8219,"mutex_wait_us":60,"spinlock_wait_cycles":1280,"thread_start_us":19263,"threads_started":1}
I20260809 15:14:08.599695 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.544551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:37083 (local address 127.15.11.189:38029)
0809 15:14:08.564906 (+ 20355us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.564916 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.564941 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:14:08.565020 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.565028 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.565131 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.571852 (+ 6721us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:08.572067 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.573427 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.577403 (+ 3976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.578338 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.579311 (+ 973us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.586375 (+ 7064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:08.586384 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:08.586389 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:08.586442 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:08.592735 (+ 6293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.593720 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.593729 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.593734 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.593920 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.594392 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.594397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.594403 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.594833 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:08.598278 (+ 3445us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.598439 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.599181 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20200,"thread_start_us":20113,"threads_started":1}
I20260809 15:14:08.601156 23437 heartbeater.cc:347] Connected to a master server at 127.15.11.189:38029
I20260809 15:14:08.601559 23437 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:08.602628 23437 heartbeater.cc:511] Master 127.15.11.189:38029 requested a full tablet report, sending...
I20260809 15:14:08.605022 22722 ts_manager.cc:194] Registered new tserver with Master: 9872f3dd69b54da6aaa37bd072bb6161 (127.15.11.131:45485)
I20260809 15:14:08.607172 23436 heartbeater.cc:347] Connected to a master server at 127.15.11.190:45689
I20260809 15:14:08.607309 23436 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:08.607762 23436 heartbeater.cc:511] Master 127.15.11.190:45689 requested a full tablet report, sending...
I20260809 15:14:08.608781 22041 ts_manager.cc:194] Re-registered known tserver with Master: 9872f3dd69b54da6aaa37bd072bb6161 (127.15.11.131:45485)
I20260809 15:14:08.609984 22041 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:50659
I20260809 15:14:08.611049 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.559258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33459 (local address 127.15.11.131:50239)
0809 15:14:08.574335 (+ 15077us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:08.574353 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:08.576969 (+ 2616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:08.577572 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:08.577578 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:08.578144 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:08.578519 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.578532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.580256 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.580263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.582148 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.582163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.583355 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.583364 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.583566 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.584272 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:08.584294 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:08.587519 (+ 3225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:08.607762 (+ 20243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:08.607771 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:08.607778 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:08.608507 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:08.609145 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:08.609150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:08.609153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:08.609321 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:08.610011 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:08.610017 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:08.610179 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 15:14:08.610308 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 15:14:08.610630 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14962,"thread_start_us":14802,"threads_started":1}
I20260809 15:14:08.612251 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.553184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:50239 (local address 127.15.11.188:33459)
0809 15:14:08.563995 (+ 10811us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.564004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.574447 (+ 10443us) server_negotiation.cc:407] Connection header received
0809 15:14:08.577154 (+ 2707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.577160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.577283 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.577372 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:08.578706 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.580007 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.582326 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.583113 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.583911 (+ 798us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.587782 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:08.587791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:08.587798 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:08.587843 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:08.603005 (+ 15162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.608766 (+ 5761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.608775 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.608782 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.608999 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.609475 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.609481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.609484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.609845 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:08.611465 (+ 1620us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.611590 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.611875 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10654,"thread_start_us":10552,"threads_started":1}
I20260809 15:14:08.613782 23435 heartbeater.cc:347] Connected to a master server at 127.15.11.188:33459
I20260809 15:14:08.613926 23435 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:08.614328 23435 heartbeater.cc:511] Master 127.15.11.188:33459 requested a full tablet report, sending...
I20260809 15:14:08.615644 22927 ts_manager.cc:194] Registered new tserver with Master: 9872f3dd69b54da6aaa37bd072bb6161 (127.15.11.131:45485)
I20260809 15:14:08.630043 23444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.584837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.15.11.190:45689)
0809 15:14:08.599602 (+ 14765us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.599610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.599634 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:14:08.599696 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.599704 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.600378 (+ 674us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.600504 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:08.612215 (+ 11711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.613573 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.617704 (+ 4131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.618552 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.618681 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.622843 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:08.622849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:08.622855 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:08.622895 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:08.626495 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.627446 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.627452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.627457 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.627609 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.628194 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.628201 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.628207 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.628614 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:08.628891 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.629513 (+ 622us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.629759 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14617,"thread_start_us":21497,"threads_started":1}
I20260809 15:14:08.632308 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.593713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.15.11.188:33459)
0809 15:14:08.612567 (+ 18854us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.612572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.612588 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:14:08.612629 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.612634 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.612745 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.612833 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:08.615434 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.617028 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.619913 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.620618 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.620745 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.624249 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:08.624260 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:08.624263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:08.624292 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:08.627600 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.628551 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.628558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.628562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.628733 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.630355 (+ 1622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.630361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.630364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.630753 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:08.631009 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.631692 (+ 683us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.631932 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18766,"thread_start_us":24925,"threads_started":1}
I20260809 15:14:08.635219 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.585130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56678 (local address 127.15.11.189:38029)
0809 15:14:08.600072 (+ 14942us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.600079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.619336 (+ 19257us) server_negotiation.cc:407] Connection header received
0809 15:14:08.619518 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.619526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.619643 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.619719 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:08.621704 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.622372 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.624490 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.625191 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.625328 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.628533 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:08.628537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:08.628540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:08.628572 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:08.632157 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.633015 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.633020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.633023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.633180 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.633645 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.633650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.633653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.634018 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:08.634205 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.634746 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.634993 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14846,"thread_start_us":21125,"threads_started":1}
I20260809 15:14:08.635226 22041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:14:08.642663 22041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:14:08.673166 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.663370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:45485 (local address 127.0.0.1:35040)
0809 15:14:08.663985 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:08.663998 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:08.664075 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:08.665403 (+ 1328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:08.665415 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:08.665455 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:08.665844 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.665859 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.667888 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.667893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.670311 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.670326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.672287 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.672293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.672464 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.672505 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 15:14:08.672597 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 15:14:08.672722 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":64,"thread_start_us":850,"threads_started":1}
I20260809 15:14:08.674100 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.664038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.15.11.131:45485)
0809 15:14:08.664725 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.664732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.664758 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:14:08.664842 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.664850 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.665156 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.665394 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:08.666063 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.667749 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.670459 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.671936 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.672060 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.672319 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.673586 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.673757 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":348,"threads_started":1}
I20260809 15:14:08.674540 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.661813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58042 (local address 127.15.11.129:46349)
0809 15:14:08.664641 (+ 2828us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.664649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.664677 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:14:08.665754 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.665765 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.666026 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.666856 (+ 830us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:08.667068 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.668334 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.671702 (+ 3368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.673515 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.673665 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.673851 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.674103 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.674257 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2647,"thread_start_us":2511,"threads_started":1}
I20260809 15:14:08.674750 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.661606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:46349 (local address 127.0.0.1:58042)
0809 15:14:08.663579 (+ 1973us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:08.663616 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:08.663750 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:08.666265 (+ 2515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:08.666270 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:08.666295 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:08.666666 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.666679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.668519 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.668524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.670592 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.670602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.673738 (+ 3136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.673744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.673908 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.673942 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:14:08.674033 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:14:08.674172 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1848,"mutex_wait_us":21,"thread_start_us":1719,"threads_started":1}
I20260809 15:14:08.675921 23467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.661114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:39013 (local address 127.0.0.1:33600)
0809 15:14:08.661754 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:08.661773 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:08.661896 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:08.669565 (+ 7669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:08.669568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:08.669586 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:08.669949 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.669956 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.671662 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.671666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.673481 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.673499 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.675219 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.675225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.675410 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.675440 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:14:08.675534 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 15:14:08.675651 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":658,"threads_started":1}
I20260809 15:14:08.676693 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.661394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33600 (local address 127.15.11.130:39013)
0809 15:14:08.668837 (+ 7443us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.668845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.668871 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:14:08.668963 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.668970 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.669182 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.669374 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:08.670121 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.671530 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.673664 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.675011 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.675141 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.675331 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.676236 (+ 905us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.676417 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7276,"thread_start_us":6863,"threads_started":1}
I20260809 15:14:08.678555 23361 tablet_service.cc:1467] Processing CreateTablet for tablet ca4b277884f44ecb96394fd8b1b0031d (DEFAULT_TABLE table=test-table [id=56fe202845824653bbcdbb2212716495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:08.678589 23050 tablet_service.cc:1467] Processing CreateTablet for tablet ca4b277884f44ecb96394fd8b1b0031d (DEFAULT_TABLE table=test-table [id=56fe202845824653bbcdbb2212716495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:08.679883 23361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4b277884f44ecb96394fd8b1b0031d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:08.679967 23050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4b277884f44ecb96394fd8b1b0031d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:08.680179 23210 tablet_service.cc:1467] Processing CreateTablet for tablet ca4b277884f44ecb96394fd8b1b0031d (DEFAULT_TABLE table=test-table [id=56fe202845824653bbcdbb2212716495]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:08.681450 23210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca4b277884f44ecb96394fd8b1b0031d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:08.693353 23473 tablet_bootstrap.cc:492] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: Bootstrap starting.
I20260809 15:14:08.696208 23474 tablet_bootstrap.cc:492] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: Bootstrap starting.
I20260809 15:14:08.697340 23475 tablet_bootstrap.cc:492] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: Bootstrap starting.
I20260809 15:14:08.697399 23473 tablet_bootstrap.cc:654] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:08.699249 23473 log.cc:824] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:08.700206 23475 tablet_bootstrap.cc:654] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:08.700376 23474 tablet_bootstrap.cc:654] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:08.701301 23475 log.cc:824] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:08.702154 23474 log.cc:824] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:08.703800 23473 tablet_bootstrap.cc:492] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: No bootstrap required, opened a new log
I20260809 15:14:08.704165 23473 ts_tablet_manager.cc:1397] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.009s
I20260809 15:14:08.705135 23474 tablet_bootstrap.cc:492] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: No bootstrap required, opened a new log
I20260809 15:14:08.705439 23474 ts_tablet_manager.cc:1397] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260809 15:14:08.706039 23475 tablet_bootstrap.cc:492] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: No bootstrap required, opened a new log
I20260809 15:14:08.706300 23475 ts_tablet_manager.cc:1397] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260809 15:14:08.714776 23474 raft_consensus.cc:348] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } }
I20260809 15:14:08.714943 23475 raft_consensus.cc:348] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } }
I20260809 15:14:08.715193 23474 raft_consensus.cc:374] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:08.715375 23475 raft_consensus.cc:374] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:08.715364 23474 raft_consensus.cc:729] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee582d5158d04b8288b8207d1593112e, State: Initialized, Role: FOLLOWER
I20260809 15:14:08.715489 23475 raft_consensus.cc:729] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ccdf564425469096c2e1f342c5d106, State: Initialized, Role: FOLLOWER
I20260809 15:14:08.715890 23473 raft_consensus.cc:348] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } }
I20260809 15:14:08.716347 23473 raft_consensus.cc:374] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:08.716071 23474 consensus_queue.cc:260] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } }
I20260809 15:14:08.716207 23475 consensus_queue.cc:260] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } }
I20260809 15:14:08.716446 23473 raft_consensus.cc:729] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9872f3dd69b54da6aaa37bd072bb6161, State: Initialized, Role: FOLLOWER
I20260809 15:14:08.717058 23473 consensus_queue.cc:260] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } }
I20260809 15:14:08.719257 23473 ts_tablet_manager.cc:1428] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260809 15:14:08.721197 23436 heartbeater.cc:503] Master 127.15.11.190:45689 was elected leader, sending a full tablet report...
I20260809 15:14:08.722522 23282 heartbeater.cc:503] Master 127.15.11.190:45689 was elected leader, sending a full tablet report...
I20260809 15:14:08.723639 23475 ts_tablet_manager.cc:1428] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260809 15:14:08.724972 23474 ts_tablet_manager.cc:1428] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: Time spent starting tablet: real 0.019s user 0.010s sys 0.005s
W20260809 15:14:08.796797 23124 tablet.cc:2369] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:14:08.825569 23438 tablet.cc:2369] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:14:08.851835 23482 raft_consensus.cc:482] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:08.852104 23482 raft_consensus.cc:504] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } }
I20260809 15:14:08.853992 23482 leader_election.cc:288] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ccdf564425469096c2e1f342c5d106 (127.15.11.130:39013), ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349)
I20260809 15:14:08.860073 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.854460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:53137 (local address 127.15.11.129:46349)
0809 15:14:08.854657 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.854663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.855015 (+ 352us) server_negotiation.cc:407] Connection header received
0809 15:14:08.855186 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.855191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.855265 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.855362 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:08.855797 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.856568 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.858240 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.859073 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.859182 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.859232 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.859674 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.859792 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:14:08.860198 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.854105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:39013 (local address 127.15.11.131:60311)
0809 15:14:08.854654 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:08.854672 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:08.854781 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:08.855203 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:08.855207 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:08.855232 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:08.855513 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.855532 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.856689 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.856696 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.858089 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.858101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.859402 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.859406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.859552 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.859631 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 15:14:08.859725 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 15:14:08.859849 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":52}
I20260809 15:14:08.860419 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.854650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:60311 (local address 127.15.11.130:39013)
0809 15:14:08.854848 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.854856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.854890 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:14:08.854992 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.854999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.855076 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.855170 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:08.855699 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.856551 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.858266 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.859251 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.859398 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.859467 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.859990 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.860111 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:14:08.860286 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.854385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:46349 (local address 127.15.11.131:53137)
0809 15:14:08.854889 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:08.854929 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:08.855044 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:08.855402 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:08.855405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:08.855426 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:08.855662 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.855674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.856722 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.856726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.858081 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:08.858094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.859236 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.859242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:08.859452 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.859496 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 15:14:08.859639 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 15:14:08.859770 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":23}
I20260809 15:14:08.861470 23232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4b277884f44ecb96394fd8b1b0031d" candidate_uuid: "9872f3dd69b54da6aaa37bd072bb6161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ccdf564425469096c2e1f342c5d106" is_pre_election: true
I20260809 15:14:08.861815 23073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4b277884f44ecb96394fd8b1b0031d" candidate_uuid: "9872f3dd69b54da6aaa37bd072bb6161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee582d5158d04b8288b8207d1593112e" is_pre_election: true
I20260809 15:14:08.861979 23232 raft_consensus.cc:2454] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9872f3dd69b54da6aaa37bd072bb6161 in term 0.
I20260809 15:14:08.862319 23073 raft_consensus.cc:2454] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9872f3dd69b54da6aaa37bd072bb6161 in term 0.
I20260809 15:14:08.862700 23313 leader_election.cc:302] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89ccdf564425469096c2e1f342c5d106, 9872f3dd69b54da6aaa37bd072bb6161; no voters:
I20260809 15:14:08.863193 23482 raft_consensus.cc:2781] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:14:08.863350 23482 raft_consensus.cc:482] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:14:08.863492 23482 raft_consensus.cc:3037] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:08.865912 23482 raft_consensus.cc:504] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } }
I20260809 15:14:08.866523 23482 leader_election.cc:288] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [CANDIDATE]: Term 1 election: Requested vote from peers 89ccdf564425469096c2e1f342c5d106 (127.15.11.130:39013), ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349)
I20260809 15:14:08.866927 23232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4b277884f44ecb96394fd8b1b0031d" candidate_uuid: "9872f3dd69b54da6aaa37bd072bb6161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ccdf564425469096c2e1f342c5d106"
I20260809 15:14:08.866976 23073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca4b277884f44ecb96394fd8b1b0031d" candidate_uuid: "9872f3dd69b54da6aaa37bd072bb6161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee582d5158d04b8288b8207d1593112e"
I20260809 15:14:08.867129 23232 raft_consensus.cc:3037] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:08.867166 23073 raft_consensus.cc:3037] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:08.869433 23073 raft_consensus.cc:2454] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9872f3dd69b54da6aaa37bd072bb6161 in term 1.
I20260809 15:14:08.869690 23232 raft_consensus.cc:2454] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9872f3dd69b54da6aaa37bd072bb6161 in term 1.
I20260809 15:14:08.869828 23314 leader_election.cc:302] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9872f3dd69b54da6aaa37bd072bb6161, ee582d5158d04b8288b8207d1593112e; no voters:
I20260809 15:14:08.870126 23482 raft_consensus.cc:2781] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:08.870599 23482 raft_consensus.cc:686] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 LEADER]: Becoming Leader. State: Replica: 9872f3dd69b54da6aaa37bd072bb6161, State: Running, Role: LEADER
I20260809 15:14:08.871173 23482 consensus_queue.cc:237] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } }
I20260809 15:14:08.876628 22038 catalog_manager.cc:5495] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9872f3dd69b54da6aaa37bd072bb6161 (127.15.11.131). New cstate: current_term: 1 leader_uuid: "9872f3dd69b54da6aaa37bd072bb6161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } health_report { overall_health: HEALTHY } } }
I20260809 15:14:08.912005 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.907176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.15.11.131:45485)
0809 15:14:08.907431 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.907435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.907453 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:14:08.907621 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.907625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.907699 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.907813 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:14:08.908178 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.908885 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.910125 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.910509 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.910594 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.911035 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:14:08.911488 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:14:08.911590 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.911660 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.911719 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 15:14:08.914148 23284 tablet.cc:2369] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:14:08.920747 23232 raft_consensus.cc:1261] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [term 1 FOLLOWER]: Refusing update from remote peer 9872f3dd69b54da6aaa37bd072bb6161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:08.920845 23073 raft_consensus.cc:1261] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [term 1 FOLLOWER]: Refusing update from remote peer 9872f3dd69b54da6aaa37bd072bb6161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:08.921718 23483 consensus_queue.cc:1035] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:08.922106 23482 consensus_queue.cc:1035] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:08.933446 23487 mvcc.cc:205] Tried to move back new op lower bound from 7316637486769033216 to 7316637486580375552. Current Snapshot: MvccSnapshot[applied={T|T < 7316637486769033216}]
I20260809 15:14:08.936416 23488 mvcc.cc:205] Tried to move back new op lower bound from 7316637486769033216 to 7316637486580375552. Current Snapshot: MvccSnapshot[applied={T|T < 7316637486769033216}]
I20260809 15:14:08.942519 23489 mvcc.cc:205] Tried to move back new op lower bound from 7316637486769033216 to 7316637486580375552. Current Snapshot: MvccSnapshot[applied={T|T < 7316637486769033216}]
I20260809 15:14:08.953071 22038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:08.953353 22038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:08.960937 22038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca6f4feb52f942af9223513fffac1276: Sending DeleteTablet for 3 replicas of tablet ca4b277884f44ecb96394fd8b1b0031d
I20260809 15:14:08.962556 23210 tablet_service.cc:1514] Processing DeleteTablet for tablet ca4b277884f44ecb96394fd8b1b0031d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:08 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:33600
I20260809 15:14:08.962706 23361 tablet_service.cc:1514] Processing DeleteTablet for tablet ca4b277884f44ecb96394fd8b1b0031d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:08 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:35040
I20260809 15:14:08.963555 23496 tablet_replica.cc:330] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: stopping tablet replica
I20260809 15:14:08.964051 23496 raft_consensus.cc:2229] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:14:08.964143 23050 tablet_service.cc:1514] Processing DeleteTablet for tablet ca4b277884f44ecb96394fd8b1b0031d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:08 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:58042
I20260809 15:14:08.964792 23496 raft_consensus.cc:2258] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:08.965158 23498 tablet_replica.cc:330] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: stopping tablet replica
I20260809 15:14:08.965634 23498 raft_consensus.cc:2229] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:08.966217 23498 raft_consensus.cc:2258] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:08.969666 23498 ts_tablet_manager.cc:1905] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:08.969734 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 22006
I20260809 15:14:08.970814 22072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:08.972599 23496 ts_tablet_manager.cc:1905] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:08.979354 23498 ts_tablet_manager.cc:1918] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:08.979605 23498 log.cc:1196] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/ca4b277884f44ecb96394fd8b1b0031d
I20260809 15:14:08.979959 23496 ts_tablet_manager.cc:1918] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:08.980432 23498 ts_tablet_manager.cc:1939] T ca4b277884f44ecb96394fd8b1b0031d P ee582d5158d04b8288b8207d1593112e: Deleting consensus metadata
I20260809 15:14:08.983783 23497 tablet_replica.cc:330] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: stopping tablet replica
I20260809 15:14:08.984850 23497 raft_consensus.cc:2229] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:08.985416 23497 raft_consensus.cc:2258] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:08.985796 23496 log.cc:1196] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/ca4b277884f44ecb96394fd8b1b0031d
I20260809 15:14:08.986506 23496 ts_tablet_manager.cc:1939] T ca4b277884f44ecb96394fd8b1b0031d P 9872f3dd69b54da6aaa37bd072bb6161: Deleting consensus metadata
I20260809 15:14:08.990185 23497 ts_tablet_manager.cc:1905] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:08.995065 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.967506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.15.11.189:38029)
0809 15:14:08.967675 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.967681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.967700 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:14:08.968931 (+ 1231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.968937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.969029 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.969133 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:08.980293 (+ 11160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.981521 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.983270 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:08.983948 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:08.984068 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:08.987928 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:08.987935 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:08.987939 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:08.988002 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:08.991601 (+ 3599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.992580 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.992585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.992589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.992741 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:08.993229 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:08.993236 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:08.993239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:08.993612 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:08.993847 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:08.994480 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 15:14:08.994703 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:14:08.999409 23497 ts_tablet_manager.cc:1918] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:08.999580 23497 log.cc:1196] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/ca4b277884f44ecb96394fd8b1b0031d
I20260809 15:14:09.000168 23497 ts_tablet_manager.cc:1939] T ca4b277884f44ecb96394fd8b1b0031d P 89ccdf564425469096c2e1f342c5d106: Deleting consensus metadata
I20260809 15:14:09.011125 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:08.968662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.15.11.188:33459)
0809 15:14:08.968828 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:08.968834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:08.996331 (+ 27497us) server_negotiation.cc:407] Connection header received
0809 15:14:08.996651 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:08.996658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:08.996746 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:08.996838 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:08.998871 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:09.000032 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:09.001967 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:09.002407 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:09.002479 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:09.005372 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:09.005379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:09.005383 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:09.005428 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:09.008363 (+ 2935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:09.009065 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:09.009070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:09.009072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:09.009201 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:09.009610 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:09.009614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:09.009616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:09.009915 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:09.010182 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:09.010504 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 15:14:09.010730 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:14:09.153031 22025 catalog_manager.cc:4841] TS 89ccdf564425469096c2e1f342c5d106 (127.15.11.130:39013): tablet ca4b277884f44ecb96394fd8b1b0031d (table test-table [id=56fe202845824653bbcdbb2212716495]) successfully deleted
I20260809 15:14:09.153069 22026 catalog_manager.cc:4841] TS ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349): tablet ca4b277884f44ecb96394fd8b1b0031d (table test-table [id=56fe202845824653bbcdbb2212716495]) successfully deleted
I20260809 15:14:09.153960 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 22006
I20260809 15:14:09.203449 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:09.184612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.15.11.189:38029)
0809 15:14:09.184879 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:09.184888 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:09.185909 (+ 1021us) server_negotiation.cc:407] Connection header received
0809 15:14:09.186244 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:09.186255 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:09.186377 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:09.186473 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:09.189351 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:09.190794 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:09.193407 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:09.194155 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:09.194253 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:09.197948 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:09.197955 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:09.197959 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:09.197998 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:09.200709 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:09.201432 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:09.201440 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:09.201446 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:09.201598 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:09.201965 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:09.201970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:09.201973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:09.202281 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:09.202581 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:09.202904 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 15:14:09.203119 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 15:14:09.206848 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:09.185672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60652 (local address 127.15.11.188:33459)
0809 15:14:09.185854 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:09.185872 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:09.191703 (+ 5831us) server_negotiation.cc:407] Connection header received
0809 15:14:09.192018 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:09.192025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:09.192105 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:09.192201 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:09.195099 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:09.195896 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:09.197330 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:09.197848 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:09.197979 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:09.200783 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:09.200791 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:09.200797 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:09.200835 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:09.204183 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:09.204894 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:09.204903 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:09.204907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:09.205058 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:09.205431 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:09.205436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:09.205440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:09.205743 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:09.206001 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:09.206325 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 15:14:09.206555 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 15:14:09.945806 23508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:09.944864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:50829)
0809 15:14:09.945459 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:09.945578 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":353,"threads_started":1}
W20260809 15:14:09.946465 23282 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:45689 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:09.949294 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:09.948356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.129:55331)
0809 15:14:09.948896 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:09.949053 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":307,"threads_started":1}
W20260809 15:14:09.949941 23122 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:45689 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:09.950387 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:09.949223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.131:55627)
0809 15:14:09.950048 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:09.950215 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":760,"threads_started":1}
W20260809 15:14:09.950744 23436 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:45689 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:10.460704 23511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ca6f4feb52f942af9223513fffac1276)
I20260809 15:14:10.460927 23511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } }
I20260809 15:14:10.461625 23512 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ca6f4feb52f942af9223513fffac1276)
I20260809 15:14:10.461930 23511 leader_election.cc:288] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689), fa97ebad4c6d41db9ca14369f747596d (127.15.11.188:33459)
I20260809 15:14:10.461882 23512 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } }
I20260809 15:14:10.463024 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.462126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36372)
0809 15:14:10.462541 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.462739 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":193,"threads_started":1}
W20260809 15:14:10.464344 22700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:10.465087 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.463891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36386)
0809 15:14:10.464529 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.464755 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":346,"threads_started":1}
W20260809 15:14:10.465438 22911 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:10.465749 23512 leader_election.cc:288] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689), 67ce804ea84f4b63999099f6206ede68 (127.15.11.189:38029)
I20260809 15:14:10.471357 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.462705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33459 (local address 127.0.0.1:60668)
0809 15:14:10.463312 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.463331 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:10.463481 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:10.464503 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:10.464509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:10.464561 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:10.464869 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:10.464888 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:10.466777 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:10.466782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:10.468980 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:10.468994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:10.470575 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:10.470583 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:10.470762 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:10.470797 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 15:14:10.470891 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 15:14:10.471026 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":22,"thread_start_us":1266,"threads_started":1}
I20260809 15:14:10.472426 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.463053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.15.11.188:33459)
0809 15:14:10.464058 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:10.464064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:10.464085 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:14:10.464186 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:10.464194 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:10.464309 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:10.465050 (+ 741us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:10.465328 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:10.466628 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:10.469173 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:10.470383 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:10.471771 (+ 1388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:10.471879 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:10.471931 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 15:14:10.472062 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":690,"threads_started":1}
I20260809 15:14:10.473176 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.464867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:38029 (local address 127.0.0.1:56698)
0809 15:14:10.465374 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.465393 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:10.465512 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:10.466617 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:10.466621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:10.466646 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:10.466970 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:10.466982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:10.468879 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:10.468884 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:10.470982 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:10.470996 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:10.472631 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:10.472637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:10.472823 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:10.472847 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:14:10.472902 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 15:14:10.472988 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"threads_started":1}
I20260809 15:14:10.474157 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.464886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56698 (local address 127.15.11.189:38029)
0809 15:14:10.466315 (+ 1429us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:10.466320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:10.466338 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:14:10.466399 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:10.466404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:10.466463 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:10.467102 (+ 639us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:10.467376 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:10.468718 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:10.471159 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:10.472424 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:10.473421 (+ 997us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:10.473558 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:10.473613 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 15:14:10.473744 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1307,"thread_start_us":1137,"threads_started":1}
I20260809 15:14:10.474406 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.473987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36388)
0809 15:14:10.474131 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.474235 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 15:14:10.474670 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.474201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36390)
0809 15:14:10.474435 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.474520 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260809 15:14:10.474685 22700 leader_election.cc:334] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689): Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
W20260809 15:14:10.474903 22911 leader_election.cc:334] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689): Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:10.475396 22732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa97ebad4c6d41db9ca14369f747596d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "67ce804ea84f4b63999099f6206ede68" is_pre_election: true
I20260809 15:14:10.475488 22937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67ce804ea84f4b63999099f6206ede68" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fa97ebad4c6d41db9ca14369f747596d" is_pre_election: true
I20260809 15:14:10.475725 22732 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa97ebad4c6d41db9ca14369f747596d in term 1.
I20260809 15:14:10.475811 22937 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67ce804ea84f4b63999099f6206ede68 in term 1.
I20260809 15:14:10.476236 22913 leader_election.cc:302] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67ce804ea84f4b63999099f6206ede68, fa97ebad4c6d41db9ca14369f747596d; no voters: ca6f4feb52f942af9223513fffac1276
I20260809 15:14:10.476586 23512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:14:10.476706 23512 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Starting leader election (detected failure of leader ca6f4feb52f942af9223513fffac1276)
I20260809 15:14:10.476816 23512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:14:10.477027 22700 leader_election.cc:302] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67ce804ea84f4b63999099f6206ede68, fa97ebad4c6d41db9ca14369f747596d; no voters: ca6f4feb52f942af9223513fffac1276
I20260809 15:14:10.477312 23511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:14:10.477448 23511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ca6f4feb52f942af9223513fffac1276)
I20260809 15:14:10.477555 23511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:14:10.479215 23512 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } }
I20260809 15:14:10.479854 23512 leader_election.cc:288] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [CANDIDATE]: Term 2 election: Requested vote from peers ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689), 67ce804ea84f4b63999099f6206ede68 (127.15.11.189:38029)
I20260809 15:14:10.479705 23511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } }
I20260809 15:14:10.480391 23511 leader_election.cc:288] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 2 election: Requested vote from peers ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689), fa97ebad4c6d41db9ca14369f747596d (127.15.11.188:33459)
I20260809 15:14:10.480504 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.480017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36398)
0809 15:14:10.480268 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.480351 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260809 15:14:10.480863 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.480487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36414)
0809 15:14:10.480635 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.480718 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 15:14:10.481017 22937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67ce804ea84f4b63999099f6206ede68" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fa97ebad4c6d41db9ca14369f747596d"
I20260809 15:14:10.481320 22937 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 67ce804ea84f4b63999099f6206ede68 in current term 2: Already voted for candidate fa97ebad4c6d41db9ca14369f747596d in this term.
I20260809 15:14:10.481274 22732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa97ebad4c6d41db9ca14369f747596d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "67ce804ea84f4b63999099f6206ede68"
I20260809 15:14:10.481549 22732 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fa97ebad4c6d41db9ca14369f747596d in current term 2: Already voted for candidate 67ce804ea84f4b63999099f6206ede68 in this term.
I20260809 15:14:10.481734 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.481204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36418)
0809 15:14:10.481430 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.481516 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260809 15:14:10.481925 22911 leader_election.cc:334] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689): Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:10.482223 22913 leader_election.cc:302] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa97ebad4c6d41db9ca14369f747596d; no voters: 67ce804ea84f4b63999099f6206ede68, ca6f4feb52f942af9223513fffac1276
I20260809 15:14:10.482556 23512 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:14:10.482885 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.482502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36426)
0809 15:14:10.482644 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.482726 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 15:14:10.483112 22700 leader_election.cc:334] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689): Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:10.483309 22700 leader_election.cc:302] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67ce804ea84f4b63999099f6206ede68; no voters: ca6f4feb52f942af9223513fffac1276, fa97ebad4c6d41db9ca14369f747596d
I20260809 15:14:10.483563 23511 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:14:10.947995 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.947165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:58797)
0809 15:14:10.947633 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.947721 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":277,"threads_started":1}
I20260809 15:14:10.952315 23528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.951528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.131:44735)
0809 15:14:10.951970 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.952086 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":239,"threads_started":1}
I20260809 15:14:10.952737 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:10.950782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.129:59941)
0809 15:14:10.952469 (+ 1687us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:10.952569 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1581,"thread_start_us":1486,"threads_started":1}
I20260809 15:14:11.949672 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:11.948847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:42635)
0809 15:14:11.949370 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:11.949490 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":267,"threads_started":1}
I20260809 15:14:11.954376 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:11.953280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.131:54221)
0809 15:14:11.954126 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:11.954219 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":649,"threads_started":1}
I20260809 15:14:11.954421 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:11.953535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.129:55895)
0809 15:14:11.954058 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:11.954185 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":306,"threads_started":1}
I20260809 15:14:12.398909 23532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:12.399142 23532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } }
I20260809 15:14:12.399737 23532 leader_election.cc:288] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689), fa97ebad4c6d41db9ca14369f747596d (127.15.11.188:33459)
I20260809 15:14:12.400833 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.399913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36700)
0809 15:14:12.400458 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.400585 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":278,"threads_started":1}
I20260809 15:14:12.400873 22937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67ce804ea84f4b63999099f6206ede68" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fa97ebad4c6d41db9ca14369f747596d" is_pre_election: true
I20260809 15:14:12.401124 22937 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67ce804ea84f4b63999099f6206ede68 in term 2.
I20260809 15:14:12.401607 22700 leader_election.cc:302] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67ce804ea84f4b63999099f6206ede68, fa97ebad4c6d41db9ca14369f747596d; no voters:
I20260809 15:14:12.401873 23532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:14:12.402001 23532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:14:12.402112 23532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:14:12.402380 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.401993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36710)
0809 15:14:12.402150 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.402237 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 15:14:12.402542 22700 leader_election.cc:334] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689): Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:12.404358 23532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } }
I20260809 15:14:12.404858 23532 leader_election.cc:288] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 3 election: Requested vote from peers ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689), fa97ebad4c6d41db9ca14369f747596d (127.15.11.188:33459)
I20260809 15:14:12.405319 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.404938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36726)
0809 15:14:12.405094 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.405191 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 15:14:12.405476 22937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67ce804ea84f4b63999099f6206ede68" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fa97ebad4c6d41db9ca14369f747596d"
I20260809 15:14:12.405716 22937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:14:12.406183 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.405903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36740)
0809 15:14:12.406018 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.406096 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 15:14:12.406419 22700 leader_election.cc:334] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689): Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
I20260809 15:14:12.408021 22937 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67ce804ea84f4b63999099f6206ede68 in term 3.
I20260809 15:14:12.408424 22700 leader_election.cc:302] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 67ce804ea84f4b63999099f6206ede68, fa97ebad4c6d41db9ca14369f747596d; no voters: ca6f4feb52f942af9223513fffac1276
I20260809 15:14:12.408746 23532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:14:12.409117 23532 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [term 3 LEADER]: Becoming Leader. State: Replica: 67ce804ea84f4b63999099f6206ede68, State: Running, Role: LEADER
I20260809 15:14:12.409485 23532 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } }
I20260809 15:14:12.411008 23535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67ce804ea84f4b63999099f6206ede68. Latest consensus state: current_term: 3 leader_uuid: "67ce804ea84f4b63999099f6206ede68" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:12.411412 23535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:12.412046 23538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:14:12.415463 23538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:14:12.416323 23538 catalog_manager.cc:1259] Loaded cluster ID: 5fc86ddb90b045fbac39a635df58f458
I20260809 15:14:12.416425 23538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:14:12.417757 23538 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:14:12.418639 23538 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Loaded TSK: 0
I20260809 15:14:12.419498 23538 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:14:12.478103 22721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:14:12.485379 22721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:14:12.489655 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.489171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36756)
0809 15:14:12.489333 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.489504 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:14:12.490404 22937 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [term 3 FOLLOWER]: Refusing update from remote peer 67ce804ea84f4b63999099f6206ede68: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 15:14:12.490917 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.490409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36770)
0809 15:14:12.490646 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.490746 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 15:14:12.491148 23532 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 15:14:12.491442 22700 consensus_peers.cc:487] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 -> Peer ca6f4feb52f942af9223513fffac1276 (127.15.11.190:45689): Couldn't send request to peer ca6f4feb52f942af9223513fffac1276. Status: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:14:12.494831 23541 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67ce804ea84f4b63999099f6206ede68. Latest consensus state: current_term: 3 leader_uuid: "67ce804ea84f4b63999099f6206ede68" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:12.495304 23541 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:14:12.495932 23532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "67ce804ea84f4b63999099f6206ede68" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:12.496371 23532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:12.496721 23541 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "67ce804ea84f4b63999099f6206ede68" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ca6f4feb52f942af9223513fffac1276" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45689 } } peers { permanent_uuid: "67ce804ea84f4b63999099f6206ede68" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 38029 } attrs { promote: false } } peers { permanent_uuid: "fa97ebad4c6d41db9ca14369f747596d" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33459 } attrs { promote: false } } }
I20260809 15:14:12.497021 23541 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa97ebad4c6d41db9ca14369f747596d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:14:12.525286 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.505310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.15.11.131:45485)
0809 15:14:12.505799 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:12.505803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:12.516866 (+ 11063us) server_negotiation.cc:407] Connection header received
0809 15:14:12.517438 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:12.517446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:12.517555 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:12.517688 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:12.518217 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.519268 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.521396 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.522649 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.522791 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.522925 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:12.524766 (+ 1841us) server_negotiation.cc:299] Negotiation successful
0809 15:14:12.524917 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":273,"threads_started":1}
I20260809 15:14:12.525274 23547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.505098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:45485 (local address 127.0.0.1:35054)
0809 15:14:12.516730 (+ 11632us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.516755 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:12.517207 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:12.517705 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:12.517709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:12.517731 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:12.518017 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.518025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.519398 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.519406 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.521185 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.521202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.524301 (+ 3099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.524306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.524431 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.524454 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:14:12.524752 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 15:14:12.524904 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11455,"thread_start_us":11902,"threads_started":1}
I20260809 15:14:12.526618 23361 tablet_service.cc:1467] Processing CreateTablet for tablet 197e36070d0844d39973285d992fa1e4 (DEFAULT_TABLE table=test-table [id=3498cfff244e421383099a5cb79bc4ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:12.526960 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.504812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:39013 (local address 127.0.0.1:33612)
0809 15:14:12.505053 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.505087 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:12.505233 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:12.518687 (+ 13454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:12.518692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:12.518716 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:12.519020 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.519030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.522708 (+ 3678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.522716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.524088 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.524102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.526174 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.526181 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.526387 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.526425 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:14:12.526496 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 15:14:12.526630 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 15:14:12.527213 23361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197e36070d0844d39973285d992fa1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:12.527974 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.504963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33612 (local address 127.15.11.130:39013)
0809 15:14:12.518362 (+ 13399us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:12.518369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:12.518388 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:14:12.518455 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:12.518461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:12.518531 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:12.519167 (+ 636us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:12.521422 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.522531 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.524323 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.525918 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.527365 (+ 1447us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.527465 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:12.527513 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 15:14:12.527648 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13251,"thread_start_us":12573,"threads_started":1}
I20260809 15:14:12.528733 23210 tablet_service.cc:1467] Processing CreateTablet for tablet 197e36070d0844d39973285d992fa1e4 (DEFAULT_TABLE table=test-table [id=3498cfff244e421383099a5cb79bc4ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:12.529196 23210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197e36070d0844d39973285d992fa1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:12.530892 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.507448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58044 (local address 127.15.11.129:46349)
0809 15:14:12.523252 (+ 15804us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:12.523256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:12.523280 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:14:12.523339 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:12.523342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:12.523397 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:12.523516 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:12.524152 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.525383 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.527893 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.529262 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.529422 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.529539 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:12.530374 (+ 835us) server_negotiation.cc:299] Negotiation successful
0809 15:14:12.530534 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15675,"thread_start_us":16195,"threads_started":1}
I20260809 15:14:12.531447 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.507276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:46349 (local address 127.0.0.1:58044)
0809 15:14:12.521621 (+ 14345us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.521645 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:12.521810 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:12.523531 (+ 1721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:12.523535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:12.523560 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:12.523955 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.523965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.525566 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.525572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.527706 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.527721 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.529703 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.529710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.529930 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.529970 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 15:14:12.530071 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 15:14:12.530204 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14186,"thread_start_us":14637,"threads_started":1}
I20260809 15:14:12.534823 23552 tablet_bootstrap.cc:492] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: Bootstrap starting.
I20260809 15:14:12.535645 23553 tablet_bootstrap.cc:492] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: Bootstrap starting.
I20260809 15:14:12.536026 23050 tablet_service.cc:1467] Processing CreateTablet for tablet 197e36070d0844d39973285d992fa1e4 (DEFAULT_TABLE table=test-table [id=3498cfff244e421383099a5cb79bc4ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:12.536502 23050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 197e36070d0844d39973285d992fa1e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:12.536903 23552 tablet_bootstrap.cc:654] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:12.540551 23553 tablet_bootstrap.cc:654] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:12.542467 23552 tablet_bootstrap.cc:492] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: No bootstrap required, opened a new log
I20260809 15:14:12.542622 23552 ts_tablet_manager.cc:1397] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 15:14:12.543440 23552 raft_consensus.cc:348] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.543701 23552 raft_consensus.cc:374] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:12.543844 23552 raft_consensus.cc:729] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9872f3dd69b54da6aaa37bd072bb6161, State: Initialized, Role: FOLLOWER
I20260809 15:14:12.544266 23556 tablet_bootstrap.cc:492] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: Bootstrap starting.
I20260809 15:14:12.544265 23552 consensus_queue.cc:260] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.546123 23556 tablet_bootstrap.cc:654] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:12.553292 23553 tablet_bootstrap.cc:492] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: No bootstrap required, opened a new log
I20260809 15:14:12.553507 23553 ts_tablet_manager.cc:1397] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.000s
I20260809 15:14:12.554312 23552 ts_tablet_manager.cc:1428] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260809 15:14:12.554443 23556 tablet_bootstrap.cc:492] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: No bootstrap required, opened a new log
I20260809 15:14:12.554602 23556 ts_tablet_manager.cc:1397] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260809 15:14:12.555344 23556 raft_consensus.cc:348] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.555567 23556 raft_consensus.cc:374] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:12.555670 23556 raft_consensus.cc:729] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee582d5158d04b8288b8207d1593112e, State: Initialized, Role: FOLLOWER
I20260809 15:14:12.555495 23553 raft_consensus.cc:348] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.556046 23553 raft_consensus.cc:374] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:12.555974 23556 consensus_queue.cc:260] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.556241 23553 raft_consensus.cc:729] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ccdf564425469096c2e1f342c5d106, State: Initialized, Role: FOLLOWER
I20260809 15:14:12.556581 23553 consensus_queue.cc:260] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.556916 23556 ts_tablet_manager.cc:1428] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:14:12.557260 23437 heartbeater.cc:503] Master 127.15.11.189:38029 was elected leader, sending a full tablet report...
I20260809 15:14:12.560029 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.545973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.131:50021)
0809 15:14:12.558332 (+ 12359us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.559735 (+ 1403us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12214,"thread_start_us":12095,"threads_started":1}
I20260809 15:14:12.561156 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.557439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.129:37415)
0809 15:14:12.560552 (+ 3113us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.560876 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1491,"thread_start_us":1841,"threads_started":1}
I20260809 15:14:12.561301 23123 heartbeater.cc:503] Master 127.15.11.189:38029 was elected leader, sending a full tablet report...
I20260809 15:14:12.561426 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.560981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.131:34847)
0809 15:14:12.561135 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.561276 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 15:14:12.562528 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.561813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.129:43781)
0809 15:14:12.562011 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.562392 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 15:14:12.562800 23553 ts_tablet_manager.cc:1428] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260809 15:14:12.564667 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.563489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:49713)
0809 15:14:12.564158 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.564383 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":338,"threads_started":1}
I20260809 15:14:12.565629 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.565240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:54431)
0809 15:14:12.565407 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.565495 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 15:14:12.821650 23561 raft_consensus.cc:482] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:12.821892 23561 raft_consensus.cc:504] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.823385 23561 leader_election.cc:288] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9872f3dd69b54da6aaa37bd072bb6161 (127.15.11.131:45485), ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349)
I20260809 15:14:12.827453 23560 raft_consensus.cc:482] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:12.827665 23560 raft_consensus.cc:504] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.829538 23560 leader_election.cc:288] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89ccdf564425469096c2e1f342c5d106 (127.15.11.130:39013), 9872f3dd69b54da6aaa37bd072bb6161 (127.15.11.131:45485)
I20260809 15:14:12.830843 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.823721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:45485 (local address 127.15.11.130:33823)
0809 15:14:12.823909 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.823929 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:12.824094 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:12.824627 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:12.824630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:12.824654 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:12.824984 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.824997 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.826247 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.826253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.828213 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.828227 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.829912 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.829918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.830184 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.830229 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 15:14:12.830384 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 15:14:12.830545 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 15:14:12.830859 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.823854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:33823 (local address 127.15.11.131:45485)
0809 15:14:12.824090 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:12.824097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:12.824120 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:14:12.824349 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:12.824356 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:12.824449 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:12.824582 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:12.825166 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.826127 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.828400 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.829602 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.829766 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.829874 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:12.830415 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 15:14:12.830554 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260809 15:14:12.830940 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.823862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:34641 (local address 127.15.11.129:46349)
0809 15:14:12.824090 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:12.824097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:12.824826 (+ 729us) server_negotiation.cc:407] Connection header received
0809 15:14:12.824966 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:12.824971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:12.825052 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:12.825151 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:12.825688 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.826618 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.828664 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.829873 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.830026 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.830152 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:12.830504 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 15:14:12.830639 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:14:12.831794 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.823745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:46349 (local address 127.15.11.130:34641)
0809 15:14:12.824702 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.824723 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:12.824838 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:12.825170 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:12.825174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:12.825197 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:12.825540 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.825553 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.826745 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.826749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.828471 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.828489 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.830073 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.830079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.830270 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.830312 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 15:14:12.831214 (+ 902us) client_negotiation.cc:240] Negotiation successful
0809 15:14:12.831366 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"mutex_wait_us":200,"thread_start_us":403,"threads_started":1}
I20260809 15:14:12.831841 23382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197e36070d0844d39973285d992fa1e4" candidate_uuid: "89ccdf564425469096c2e1f342c5d106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9872f3dd69b54da6aaa37bd072bb6161" is_pre_election: true
I20260809 15:14:12.832374 23382 raft_consensus.cc:2454] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ccdf564425469096c2e1f342c5d106 in term 0.
I20260809 15:14:12.833057 23073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197e36070d0844d39973285d992fa1e4" candidate_uuid: "89ccdf564425469096c2e1f342c5d106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee582d5158d04b8288b8207d1593112e" is_pre_election: true
I20260809 15:14:12.833282 23073 raft_consensus.cc:2454] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ccdf564425469096c2e1f342c5d106 in term 0.
I20260809 15:14:12.833761 23155 leader_election.cc:302] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89ccdf564425469096c2e1f342c5d106, 9872f3dd69b54da6aaa37bd072bb6161; no voters:
I20260809 15:14:12.834501 23561 raft_consensus.cc:2781] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:14:12.834630 23561 raft_consensus.cc:482] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:14:12.834728 23561 raft_consensus.cc:3037] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:12.837653 23561 raft_consensus.cc:504] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.838370 23561 leader_election.cc:288] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [CANDIDATE]: Term 1 election: Requested vote from peers 9872f3dd69b54da6aaa37bd072bb6161 (127.15.11.131:45485), ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349)
I20260809 15:14:12.839215 23382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197e36070d0844d39973285d992fa1e4" candidate_uuid: "89ccdf564425469096c2e1f342c5d106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9872f3dd69b54da6aaa37bd072bb6161"
I20260809 15:14:12.839195 23073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197e36070d0844d39973285d992fa1e4" candidate_uuid: "89ccdf564425469096c2e1f342c5d106" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee582d5158d04b8288b8207d1593112e"
I20260809 15:14:12.839432 23073 raft_consensus.cc:3037] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:12.839439 23382 raft_consensus.cc:3037] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:12.839655 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.830086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:33663 (local address 127.15.11.130:39013)
0809 15:14:12.830301 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:12.830309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:12.831647 (+ 1338us) server_negotiation.cc:407] Connection header received
0809 15:14:12.831916 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:12.831921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:12.831987 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:12.832049 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:12.832967 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.833965 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.836658 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.837918 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.838048 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.838165 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:12.839301 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0809 15:14:12.839419 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:14:12.839990 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.830476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:45485 (local address 127.15.11.129:54645)
0809 15:14:12.832567 (+ 2091us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.832586 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:12.832790 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:12.833335 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:12.833339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:12.833388 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:12.833705 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.833717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.834910 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.834913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.836446 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.836458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.838298 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.838303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.838988 (+ 685us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.839034 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 15:14:12.839471 (+ 437us) client_negotiation.cc:240] Negotiation successful
0809 15:14:12.839630 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1548,"mutex_wait_us":52,"thread_start_us":1332,"threads_started":1}
I20260809 15:14:12.840566 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.829754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:39013 (local address 127.15.11.129:33663)
0809 15:14:12.831554 (+ 1800us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.831572 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:12.831749 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:12.832284 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:12.832288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:12.832314 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:12.832776 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.832791 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.834139 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.834143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.836467 (+ 2324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:12.836481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.838884 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.838894 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:12.839116 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.839156 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 15:14:12.839988 (+ 832us) client_negotiation.cc:240] Negotiation successful
0809 15:14:12.840317 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1681,"mutex_wait_us":28}
I20260809 15:14:12.842478 23382 raft_consensus.cc:2454] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ccdf564425469096c2e1f342c5d106 in term 1.
I20260809 15:14:12.842505 23073 raft_consensus.cc:2454] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ccdf564425469096c2e1f342c5d106 in term 1.
I20260809 15:14:12.842877 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.830942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:54645 (local address 127.15.11.131:45485)
0809 15:14:12.831176 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:12.831181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:12.832680 (+ 1499us) server_negotiation.cc:407] Connection header received
0809 15:14:12.832987 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:12.832992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:12.833109 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:12.833155 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:12.833853 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.834726 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.836595 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.837811 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.838011 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.838082 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:12.839197 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0809 15:14:12.839339 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70,"thread_start_us":5222,"threads_started":1}
I20260809 15:14:12.842962 23158 leader_election.cc:302] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89ccdf564425469096c2e1f342c5d106, ee582d5158d04b8288b8207d1593112e; no voters:
I20260809 15:14:12.843355 23561 raft_consensus.cc:2781] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:12.843320 23232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197e36070d0844d39973285d992fa1e4" candidate_uuid: "ee582d5158d04b8288b8207d1593112e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89ccdf564425469096c2e1f342c5d106" is_pre_election: true
I20260809 15:14:12.843623 23382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "197e36070d0844d39973285d992fa1e4" candidate_uuid: "ee582d5158d04b8288b8207d1593112e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9872f3dd69b54da6aaa37bd072bb6161" is_pre_election: true
I20260809 15:14:12.843921 23382 raft_consensus.cc:2379] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ee582d5158d04b8288b8207d1593112e in current term 1: Already voted for candidate 89ccdf564425469096c2e1f342c5d106 in this term.
I20260809 15:14:12.844146 23561 raft_consensus.cc:686] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 1 LEADER]: Becoming Leader. State: Replica: 89ccdf564425469096c2e1f342c5d106, State: Running, Role: LEADER
I20260809 15:14:12.844587 23283 heartbeater.cc:503] Master 127.15.11.189:38029 was elected leader, sending a full tablet report...
I20260809 15:14:12.844816 23561 consensus_queue.cc:237] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } }
I20260809 15:14:12.845654 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.845220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:34821)
0809 15:14:12.845382 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.845509 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 15:14:12.847658 22997 leader_election.cc:302] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ee582d5158d04b8288b8207d1593112e; no voters: 89ccdf564425469096c2e1f342c5d106, 9872f3dd69b54da6aaa37bd072bb6161
I20260809 15:14:12.848057 23560 raft_consensus.cc:2726] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:14:12.849028 22722 catalog_manager.cc:5495] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89ccdf564425469096c2e1f342c5d106 (127.15.11.130). New cstate: current_term: 1 leader_uuid: "89ccdf564425469096c2e1f342c5d106" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ccdf564425469096c2e1f342c5d106" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 39013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 } health_report { overall_health: UNKNOWN } } }
I20260809 15:14:12.896183 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.890611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.15.11.130:39013)
0809 15:14:12.890811 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:12.890817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:12.890837 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:14:12.891038 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:12.891042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:12.891121 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:12.891274 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:14:12.891644 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.892615 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.893973 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.894384 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.894553 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.894950 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:14:12.895508 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:14:12.895678 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:12.895729 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 15:14:12.895811 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:14:12.900782 23073 raft_consensus.cc:1261] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 1 FOLLOWER]: Refusing update from remote peer 89ccdf564425469096c2e1f342c5d106: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:12.901010 23382 raft_consensus.cc:1261] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 FOLLOWER]: Refusing update from remote peer 89ccdf564425469096c2e1f342c5d106: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:12.901625 23568 consensus_queue.cc:1035] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee582d5158d04b8288b8207d1593112e" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 46349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:12.902027 23561 consensus_queue.cc:1035] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9872f3dd69b54da6aaa37bd072bb6161" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:12.904457 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.903903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.129:43335)
0809 15:14:12.904137 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.904268 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 15:14:12.905624 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.905229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.131:51053)
0809 15:14:12.905402 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.905501 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:14:12.912880 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.912394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.131:44771)
0809 15:14:12.912595 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.912686 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 15:14:12.915465 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.914896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.129:32795)
0809 15:14:12.915185 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.915268 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260809 15:14:12.917511 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.912844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:49169)
0809 15:14:12.917045 (+ 4201us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.917186 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4024}
I20260809 15:14:12.918807 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.918242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.15.11.130:59675)
0809 15:14:12.918513 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:12.918611 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260809 15:14:12.925233 22716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:12.925531 22716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:12.931375 22716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67ce804ea84f4b63999099f6206ede68: Sending DeleteTablet for 3 replicas of tablet 197e36070d0844d39973285d992fa1e4
I20260809 15:14:12.932498 23361 tablet_service.cc:1514] Processing DeleteTablet for tablet 197e36070d0844d39973285d992fa1e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:12 UTC) from {username='oryx', principal='oryx/127.15.11.189@KRBTEST.COM'} at 127.0.0.1:35054
I20260809 15:14:12.932739 23050 tablet_service.cc:1514] Processing DeleteTablet for tablet 197e36070d0844d39973285d992fa1e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:12 UTC) from {username='oryx', principal='oryx/127.15.11.189@KRBTEST.COM'} at 127.0.0.1:58044
I20260809 15:14:12.934204 23210 tablet_service.cc:1514] Processing DeleteTablet for tablet 197e36070d0844d39973285d992fa1e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:12 UTC) from {username='oryx', principal='oryx/127.15.11.189@KRBTEST.COM'} at 127.0.0.1:33612
I20260809 15:14:12.938362 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 22964
I20260809 15:14:12.944098 23582 tablet_replica.cc:330] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: stopping tablet replica
I20260809 15:14:12.944386 23582 raft_consensus.cc:2229] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:12.944747 23582 raft_consensus.cc:2258] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:12.946322 23582 ts_tablet_manager.cc:1905] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:12.951325 23583 tablet_replica.cc:330] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: stopping tablet replica
I20260809 15:14:12.951534 23583 raft_consensus.cc:2229] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:14:12.952307 23583 raft_consensus.cc:2258] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:12.953908 23583 ts_tablet_manager.cc:1905] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:12.953902 23582 ts_tablet_manager.cc:1918] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:12.954049 23582 log.cc:1196] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/197e36070d0844d39973285d992fa1e4
I20260809 15:14:12.954559 23582 ts_tablet_manager.cc:1939] T 197e36070d0844d39973285d992fa1e4 P ee582d5158d04b8288b8207d1593112e: Deleting consensus metadata
I20260809 15:14:12.955690 22701 catalog_manager.cc:4841] TS ee582d5158d04b8288b8207d1593112e (127.15.11.129:46349): tablet 197e36070d0844d39973285d992fa1e4 (table test-table [id=3498cfff244e421383099a5cb79bc4ae]) successfully deleted
I20260809 15:14:12.956578 23581 tablet_replica.cc:330] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: stopping tablet replica
I20260809 15:14:12.956766 23581 raft_consensus.cc:2229] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:12.957065 23581 raft_consensus.cc:2258] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:12.958349 23581 ts_tablet_manager.cc:1905] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:12.964824 23583 ts_tablet_manager.cc:1918] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:12.965232 23581 ts_tablet_manager.cc:1918] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:12.965421 23581 log.cc:1196] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/197e36070d0844d39973285d992fa1e4
I20260809 15:14:12.965461 23583 log.cc:1196] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/197e36070d0844d39973285d992fa1e4
I20260809 15:14:12.965941 23581 ts_tablet_manager.cc:1939] T 197e36070d0844d39973285d992fa1e4 P 9872f3dd69b54da6aaa37bd072bb6161: Deleting consensus metadata
I20260809 15:14:12.966169 23583 ts_tablet_manager.cc:1939] T 197e36070d0844d39973285d992fa1e4 P 89ccdf564425469096c2e1f342c5d106: Deleting consensus metadata
I20260809 15:14:12.967552 22700 catalog_manager.cc:4841] TS 89ccdf564425469096c2e1f342c5d106 (127.15.11.130:39013): tablet 197e36070d0844d39973285d992fa1e4 (table test-table [id=3498cfff244e421383099a5cb79bc4ae]) successfully deleted
I20260809 15:14:12.967917 22698 catalog_manager.cc:4841] TS 9872f3dd69b54da6aaa37bd072bb6161 (127.15.11.131:45485): tablet 197e36070d0844d39973285d992fa1e4 (table test-table [id=3498cfff244e421383099a5cb79bc4ae]) successfully deleted
I20260809 15:14:12.974309 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:12.940463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58056 (local address 127.15.11.129:46349)
0809 15:14:12.940686 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:12.940695 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:12.956165 (+ 15470us) server_negotiation.cc:407] Connection header received
0809 15:14:12.957500 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:12.957509 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:12.957616 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:12.957727 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:12.959575 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.961178 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.962949 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:12.963798 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:12.963964 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:12.968229 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:12.968245 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:12.968261 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:12.968296 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:12.971297 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:12.972003 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:12.972011 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:12.972013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:12.972215 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:12.972606 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:12.972614 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:12.972617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:12.973030 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:12.973361 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:12.973685 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 15:14:12.973954 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:14:12.975015 23116 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:13.033702 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:13.032876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36782)
0809 15:14:13.033378 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:13.033495 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":234,"threads_started":1}
I20260809 15:14:13.035243 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:13.034839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45689 (local address 127.0.0.1:36796)
0809 15:14:13.035006 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:13.035105 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:45689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 15:14:13.074450 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 22964
I20260809 15:14:13.104738 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 23133
I20260809 15:14:13.121186 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:13.105582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.15.11.130:39013)
0809 15:14:13.105781 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:13.105787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:13.105847 (+ 60us) server_negotiation.cc:407] Connection header received
0809 15:14:13.106093 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:13.106099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:13.106180 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:13.106273 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:13.108068 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:13.109229 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:13.111099 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:13.111534 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:13.111635 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:13.115122 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:13.115138 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:13.115155 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:13.115205 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:13.118310 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:13.119092 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:13.119099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:13.119101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:13.119238 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:13.119631 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:13.119634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:13.119636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:13.120027 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:13.120351 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:13.120686 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 15:14:13.120870 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:14:13.122011 23276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:13.239941 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 23133
I20260809 15:14:13.264758 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 23295
I20260809 15:14:13.277670 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:13.265564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35064 (local address 127.15.11.131:45485)
0809 15:14:13.265736 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:13.265741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:13.265810 (+ 69us) server_negotiation.cc:407] Connection header received
0809 15:14:13.265960 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:13.265963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:13.266039 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:13.266124 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:13.267389 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:13.268055 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:13.269428 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:13.269850 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:13.269962 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:13.272558 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:13.272575 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:13.272592 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:13.272632 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:13.275314 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:13.275924 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:13.275931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:13.275934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:13.276078 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:13.276449 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:13.276452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:13.276454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:13.276812 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:13.277057 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:13.277286 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 15:14:13.277445 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:14:13.278327 23429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:13.400437 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 23295
I20260809 15:14:13.427484 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 22560
I20260809 15:14:13.428663 22759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:13.554572 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 22560
W20260809 15:14:13.580977 22913 connection.cc:535] server connection from 127.0.0.1:60668 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 15:14:13.581415 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 22667
I20260809 15:14:13.582549 22957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:13.701113 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 22667
2026-08-09T15:14:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12019 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:14:13 dist-test-slave-8w9v krb5kdc[23594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:14:13 dist-test-slave-8w9v krb5kdc[23594](info): set up 2 sockets
Aug 09 15:14:13 dist-test-slave-8w9v krb5kdc[23594](info): commencing operation
krb5kdc: starting...
W20260809 15:14:14.836684 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:14:14 dist-test-slave-8w9v krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:14:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:14:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.190.keytab.
Entry for principal oryx/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.190.keytab.
I20260809 15:14:15.071722 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:38881
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.190.keytab
--principal=oryx/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:38881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:15.406277 23608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:15.406646 23608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:15.406827 23608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:15.414052 23608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:15.414215 23608 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 15:14:15.414327 23608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:15.414448 23608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:15.414538 23608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:15.414657 23608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:15.425282 23608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:38881
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.190.keytab
--principal=oryx/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:38881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:15.426614 23608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:15.427980 23608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:15.440623 23617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:15.440775 23614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:15.440670 23615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:15.441414 23608 server_base.cc:1034] running on GCE node
I20260809 15:14:15.442217 23608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:15.443550 23608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:15.444825 23608 hybrid_clock.cc:648] HybridClock initialized: now 1786288455444761 us; error 39 us; skew 500 ppm
I20260809 15:14:15.445205 23608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:15 dist-test-slave-8w9v krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288455, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:15.448338 23608 init.cc:377] Logged in from keytab as oryx/127.15.11.190@KRBTEST.COM (short username oryx)
I20260809 15:14:15.449704 23608 webserver.cc:460] Webserver started at http://127.15.11.190:40697/ using document root <none> and password file <none>
I20260809 15:14:15.450327 23608 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:15.450414 23608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:15.450675 23608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:15.453720 23608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "3fab7693283d433783e9bc39ae1b50bf"
format_stamp: "Formatted at 2026-08-09 15:14:15 on dist-test-slave-8w9v"
server_key: "3e6c3b696eaf5e0d78aecdc9683f7ff8"
server_key_iv: "9be39e6cdd1e573e5a4e5213e89f105c"
server_key_version: "encryptionkey@0"
I20260809 15:14:15.454465 23608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "3fab7693283d433783e9bc39ae1b50bf"
format_stamp: "Formatted at 2026-08-09 15:14:15 on dist-test-slave-8w9v"
server_key: "3e6c3b696eaf5e0d78aecdc9683f7ff8"
server_key_iv: "9be39e6cdd1e573e5a4e5213e89f105c"
server_key_version: "encryptionkey@0"
I20260809 15:14:15.459371 23608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:14:15.462783 23626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:15.464042 23608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:14:15.464241 23608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "3fab7693283d433783e9bc39ae1b50bf"
format_stamp: "Formatted at 2026-08-09 15:14:15 on dist-test-slave-8w9v"
server_key: "3e6c3b696eaf5e0d78aecdc9683f7ff8"
server_key_iv: "9be39e6cdd1e573e5a4e5213e89f105c"
server_key_version: "encryptionkey@0"
I20260809 15:14:15.464421 23608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:15.489360 23608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:15.493251 23608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:15.493456 23608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:15.514948 23608 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:38881
I20260809 15:14:15.514969 23693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:38881 every 8 connection(s)
I20260809 15:14:15.516369 23608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:14:15.520056 23694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:15.526232 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 23608
I20260809 15:14:15.526467 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:14:15.527390 15406 external_mini_cluster.cc:1428] Setting key 14461143448574275284e7e3421555d2
I20260809 15:14:15.530920 23694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf: Bootstrap starting.
I20260809 15:14:15.534497 23694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:15.535604 23694 log.cc:824] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:15.538683 23694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf: No bootstrap required, opened a new log
Aug 09 15:14:15 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:15.546890 23694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fab7693283d433783e9bc39ae1b50bf" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38881 } }
I20260809 15:14:15.547389 23694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:15.547535 23694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fab7693283d433783e9bc39ae1b50bf, State: Initialized, Role: FOLLOWER
I20260809 15:14:15.548260 23694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fab7693283d433783e9bc39ae1b50bf" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38881 } }
I20260809 15:14:15.548539 23694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:14:15.548676 23694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:14:15.548856 23694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:15.550930 23698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:15.529475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47018 (local address 127.15.11.190:38881)
0809 15:14:15.530233 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:15.530243 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:15.530283 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:14:15.531876 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:15.531898 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:15.532315 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:15.532605 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:15.534395 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:15.535797 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:15.537434 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:15.538066 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:15.538193 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:15.542945 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:15.542952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:15.542965 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:15.543001 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:15.545994 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:15.546699 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:15.546708 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:15.546722 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:15.546976 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:15.547433 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:15.547438 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:15.547441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:15.548078 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:15.548416 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:15.549422 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0809 15:14:15.549908 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":370,"threads_started":1}
I20260809 15:14:15.551590 23694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fab7693283d433783e9bc39ae1b50bf" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38881 } }
I20260809 15:14:15.552088 23694 leader_election.cc:302] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fab7693283d433783e9bc39ae1b50bf; no voters:
I20260809 15:14:15.552820 23694 leader_election.cc:288] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:14:15.552937 23700 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:15.553943 23700 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [term 1 LEADER]: Becoming Leader. State: Replica: 3fab7693283d433783e9bc39ae1b50bf, State: Running, Role: LEADER
I20260809 15:14:15.554529 23700 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fab7693283d433783e9bc39ae1b50bf" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38881 } }
I20260809 15:14:15.555106 23694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:14:15.557772 23701 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fab7693283d433783e9bc39ae1b50bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fab7693283d433783e9bc39ae1b50bf" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38881 } } }
I20260809 15:14:15.557813 23702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fab7693283d433783e9bc39ae1b50bf. Latest consensus state: current_term: 1 leader_uuid: "3fab7693283d433783e9bc39ae1b50bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fab7693283d433783e9bc39ae1b50bf" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 38881 } } }
I20260809 15:14:15.558192 23701 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:15.558204 23702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:15.563024 23709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:14:15.568778 23709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:14:15.577697 23709 catalog_manager.cc:1347] Generated new cluster ID: 7c22aa073de84e3a8bdbd4359a38f2ce
I20260809 15:14:15.577813 23709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:14:15.592955 23709 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:14:15.594201 23709 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:14:15.610455 23709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf: Generated new TSK 0
I20260809 15:14:15.611328 23709 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab.
Entry for principal oryx/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab.
I20260809 15:14:15.671660 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38881
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab
--principal=oryx/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:16.004271 23724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:16.004556 23724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:16.004721 23724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:16.011832 23724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:16.011937 23724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:16.012049 23724 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:14:16.012230 23724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:14:16.021821 23724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab
--principal=oryx/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.11.190:38881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:16.023061 23724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:16.024461 23724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:16.037799 23730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:16.037844 23729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:16.037885 23732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:16.038764 23724 server_base.cc:1034] running on GCE node
I20260809 15:14:16.039351 23724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:16.040280 23724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:16.041527 23724 hybrid_clock.cc:648] HybridClock initialized: now 1786288456041461 us; error 39 us; skew 500 ppm
I20260809 15:14:16.041836 23724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:16 dist-test-slave-8w9v krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288456, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:16.044991 23724 init.cc:377] Logged in from keytab as oryx/127.15.11.129@KRBTEST.COM (short username oryx)
I20260809 15:14:16.046340 23724 webserver.cc:460] Webserver started at http://127.15.11.129:38569/ using document root <none> and password file <none>
I20260809 15:14:16.046983 23724 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:16.047070 23724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:16.047335 23724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:16.050221 23724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "f0e56413686b4d6a9b2c04dab129457d"
format_stamp: "Formatted at 2026-08-09 15:14:16 on dist-test-slave-8w9v"
server_key: "7bbaf124218d3cf59ce47626ddf64feb"
server_key_iv: "f8d4e6fc51177f180097b110475a0304"
server_key_version: "encryptionkey@0"
I20260809 15:14:16.051000 23724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "f0e56413686b4d6a9b2c04dab129457d"
format_stamp: "Formatted at 2026-08-09 15:14:16 on dist-test-slave-8w9v"
server_key: "7bbaf124218d3cf59ce47626ddf64feb"
server_key_iv: "f8d4e6fc51177f180097b110475a0304"
server_key_version: "encryptionkey@0"
I20260809 15:14:16.056614 23724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260809 15:14:16.060041 23739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:16.061370 23724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:14:16.061538 23724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "f0e56413686b4d6a9b2c04dab129457d"
format_stamp: "Formatted at 2026-08-09 15:14:16 on dist-test-slave-8w9v"
server_key: "7bbaf124218d3cf59ce47626ddf64feb"
server_key_iv: "f8d4e6fc51177f180097b110475a0304"
server_key_version: "encryptionkey@0"
I20260809 15:14:16.061749 23724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:16.090083 23724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:16.094134 23724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:16.094488 23724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:16.097555 23724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:16.097707 23724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:16.097867 23724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:16.097993 23724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 09 15:14:16 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288456, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:16.123180 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:16.097629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:38201 (local address 127.15.11.190:38881)
0809 15:14:16.098219 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:16.098228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:16.099565 (+ 1337us) server_negotiation.cc:407] Connection header received
0809 15:14:16.102646 (+ 3081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:16.102651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:16.102721 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:16.102809 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:16.105532 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.106517 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.109900 (+ 3383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.110598 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.110764 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:16.115450 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:16.115459 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:16.115466 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:16.115514 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:16.119458 (+ 3944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:16.120369 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:16.120378 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:16.120381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:16.120557 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:16.121107 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:16.121113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:16.121117 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:16.121532 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:16.121687 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:16.122578 (+ 891us) server_negotiation.cc:299] Negotiation successful
0809 15:14:16.122799 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":255,"threads_started":1}
I20260809 15:14:16.124554 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:16.098305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38881 (local address 127.15.11.129:38201)
0809 15:14:16.099362 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:16.099428 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:16.102262 (+ 2834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:16.103123 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:16.103135 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:16.104116 (+ 981us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:16.105290 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:16.105331 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.106715 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.106726 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:16.109600 (+ 2874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:16.109622 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.110922 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.110931 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:16.111170 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:16.111994 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:16.112031 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:16.115208 (+ 3177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:16.119632 (+ 4424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:16.119642 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:16.119663 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:16.120199 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:16.120723 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:16.120727 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:16.120731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:16.120968 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:16.121663 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:16.121673 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:16.122190 (+ 517us) client_negotiation.cc:769] Sending connection context
0809 15:14:16.122489 (+ 299us) client_negotiation.cc:240] Negotiation successful
0809 15:14:16.123006 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":438,"threads_started":1}
I20260809 15:14:16.143345 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:16.130679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:45403 (local address 127.15.11.190:38881)
0809 15:14:16.130827 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:16.130830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:16.131705 (+ 875us) server_negotiation.cc:407] Connection header received
0809 15:14:16.131891 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:16.131896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:16.131966 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:16.132067 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:16.133335 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.134019 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.135267 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.135656 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.135789 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:16.137566 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:16.137575 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:16.137584 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:16.137627 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:16.140583 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:16.141271 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:16.141279 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:16.141283 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:16.141460 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:16.141994 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:16.142000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:16.142005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:16.142278 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:16.142439 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:16.142786 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 15:14:16.143026 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 15:14:16.143453 23831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:16.130859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38881 (local address 127.15.11.129:45403)
0809 15:14:16.131607 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:16.131623 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:16.131773 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:16.132091 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:16.132095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:16.132687 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:16.133190 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:16.133203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.134147 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.134154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:16.135121 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:16.135131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.135835 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.135841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:16.136029 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:16.136680 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:16.136697 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:16.137379 (+ 682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:16.140732 (+ 3353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:16.140737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:16.140743 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:16.141154 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:16.141601 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:16.141610 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:16.141617 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:16.141850 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:16.142396 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:16.142400 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:16.142567 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 15:14:16.142700 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 15:14:16.143060 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":436,"threads_started":1}
I20260809 15:14:16.152729 23724 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:32971
I20260809 15:14:16.152738 23867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:32971 every 8 connection(s)
I20260809 15:14:16.153999 23724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:14:16.155910 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 23724
I20260809 15:14:16.156136 23868 heartbeater.cc:347] Connected to a master server at 127.15.11.190:38881
I20260809 15:14:16.156224 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:14:16.156368 23868 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:16.156971 15406 external_mini_cluster.cc:1428] Setting key 5190db0e0ba716dfb6ce5c0cf7dc65c1
I20260809 15:14:16.157106 23868 heartbeater.cc:511] Master 127.15.11.190:38881 requested a full tablet report, sending...
I20260809 15:14:16.159063 23643 ts_manager.cc:194] Registered new tserver with Master: f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971)
I20260809 15:14:16.160507 23643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:38201
WARNING: no policy specified for oryx/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab.
Entry for principal oryx/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab.
I20260809 15:14:16.208941 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38881
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab
--principal=oryx/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:16.531574 23872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:16.531826 23872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:16.531992 23872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:16.539148 23872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:16.539283 23872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:16.539436 23872 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:14:16.539565 23872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:14:16.549145 23872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab
--principal=oryx/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.11.190:38881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:16.550513 23872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:16.552541 23872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:16.566134 23877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:16.566195 23880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:16.566221 23878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:16.566928 23872 server_base.cc:1034] running on GCE node
I20260809 15:14:16.567596 23872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:16.568672 23872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:16.569967 23872 hybrid_clock.cc:648] HybridClock initialized: now 1786288456569882 us; error 46 us; skew 500 ppm
I20260809 15:14:16.570256 23872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:16 dist-test-slave-8w9v krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288456, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:16.573473 23872 init.cc:377] Logged in from keytab as oryx/127.15.11.130@KRBTEST.COM (short username oryx)
I20260809 15:14:16.574848 23872 webserver.cc:460] Webserver started at http://127.15.11.130:43785/ using document root <none> and password file <none>
I20260809 15:14:16.575557 23872 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:16.575695 23872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:16.576031 23872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:16.579231 23872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9"
format_stamp: "Formatted at 2026-08-09 15:14:16 on dist-test-slave-8w9v"
server_key: "7cb22075d8d72a83f37e9d7c1a88caca"
server_key_iv: "ef12bc3e7ee2ba2f8100f718fa70ca9e"
server_key_version: "encryptionkey@0"
I20260809 15:14:16.580084 23872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9"
format_stamp: "Formatted at 2026-08-09 15:14:16 on dist-test-slave-8w9v"
server_key: "7cb22075d8d72a83f37e9d7c1a88caca"
server_key_iv: "ef12bc3e7ee2ba2f8100f718fa70ca9e"
server_key_version: "encryptionkey@0"
I20260809 15:14:16.585577 23872 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:14:16.589025 23888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:16.590307 23872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:14:16.590484 23872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9"
format_stamp: "Formatted at 2026-08-09 15:14:16 on dist-test-slave-8w9v"
server_key: "7cb22075d8d72a83f37e9d7c1a88caca"
server_key_iv: "ef12bc3e7ee2ba2f8100f718fa70ca9e"
server_key_version: "encryptionkey@0"
I20260809 15:14:16.590709 23872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:16.609721 23872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:16.613713 23872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:16.614007 23872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:16.616677 23872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:16.616781 23872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:16.616925 23872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:16.617005 23872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:14:16 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288456, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:16.641355 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:16.617302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:34239 (local address 127.15.11.190:38881)
0809 15:14:16.617514 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:16.617522 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:16.618980 (+ 1458us) server_negotiation.cc:407] Connection header received
0809 15:14:16.621047 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:16.621053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:16.621150 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:16.621260 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:16.623571 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.624932 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.627434 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.628231 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.628402 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:16.633818 (+ 5416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:16.633826 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:16.633832 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:16.633876 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:16.637114 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:16.638181 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:16.638185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:16.638187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:16.638297 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:16.638965 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:16.638970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:16.638973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:16.639323 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:16.639485 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:16.640614 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0809 15:14:16.640874 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:14:16.643440 23898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:16.617995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38881 (local address 127.15.11.130:34239)
0809 15:14:16.618812 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:16.618850 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:16.620731 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:16.621518 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:16.621533 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:16.622323 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:16.623329 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:16.623361 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.625191 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.625201 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:16.627240 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:16.627259 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.628494 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.628503 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:16.628848 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:16.629634 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:16.629682 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:16.633554 (+ 3872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:16.637284 (+ 3730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:16.637297 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:16.637320 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:16.638051 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:16.638424 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:16.638432 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:16.638437 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:16.638788 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:16.639495 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:16.639512 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:16.640093 (+ 581us) client_negotiation.cc:769] Sending connection context
0809 15:14:16.640491 (+ 398us) client_negotiation.cc:240] Negotiation successful
0809 15:14:16.641227 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":370,"threads_started":1}
I20260809 15:14:16.670396 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:16.653757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:58733 (local address 127.15.11.190:38881)
0809 15:14:16.653976 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:16.653984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:16.654604 (+ 620us) server_negotiation.cc:407] Connection header received
0809 15:14:16.654795 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:16.654802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:16.654897 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:16.654994 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:16.656799 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.657518 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.660062 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.660745 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.660902 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:16.663493 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:16.663501 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:16.663507 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:16.663559 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:16.667074 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:16.668154 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:16.668159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:16.668162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:16.668318 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:16.668762 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:16.668765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:16.668767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:16.669184 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:16.669287 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:16.669858 (+ 571us) server_negotiation.cc:299] Negotiation successful
0809 15:14:16.670181 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 15:14:16.670871 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:16.653895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38881 (local address 127.15.11.130:58733)
0809 15:14:16.654510 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:16.654524 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:16.654664 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:16.655071 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:16.655082 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:16.655871 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:16.656587 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:16.656604 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.657672 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.657680 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:16.659895 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:16.659916 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:16.660961 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:16.660967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:16.661162 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:16.661820 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:16.661845 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:16.663239 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:16.667245 (+ 4006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:16.667257 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:16.667264 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:16.667997 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:16.668445 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:16.668453 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:16.668459 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:16.668647 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:16.669317 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:16.669325 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:16.669588 (+ 263us) client_negotiation.cc:769] Sending connection context
0809 15:14:16.669771 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 15:14:16.670175 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":345,"threads_started":1}
I20260809 15:14:16.676803 23872 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:43443
I20260809 15:14:16.676872 24019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:43443 every 8 connection(s)
I20260809 15:14:16.678030 23872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:14:16.679973 24020 heartbeater.cc:347] Connected to a master server at 127.15.11.190:38881
I20260809 15:14:16.680222 24020 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:16.680948 24020 heartbeater.cc:511] Master 127.15.11.190:38881 requested a full tablet report, sending...
I20260809 15:14:16.681160 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 23872
I20260809 15:14:16.681334 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:14:16.681905 15406 external_mini_cluster.cc:1428] Setting key 56980a5ff2fd00a9d954b75630a2e0e0
I20260809 15:14:16.682508 23643 ts_manager.cc:194] Registered new tserver with Master: 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443)
I20260809 15:14:16.683470 23643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:34239
WARNING: no policy specified for oryx/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab.
Entry for principal oryx/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab.
I20260809 15:14:16.732592 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:38881
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab
--principal=oryx/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:17.063802 24026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:17.064105 24026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:17.064288 24026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:17.071905 24026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:17.072006 24026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:17.072089 24026 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:14:17.072230 24026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:14:17.082226 24026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab
--principal=oryx/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.11.190:38881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:17.083585 24026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:17.085204 24026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:17.098821 24031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:17.098941 24032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:17.098953 24034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:17.099843 24026 server_base.cc:1034] running on GCE node
I20260809 15:14:17.100502 24026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:17.101464 24026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:17.102679 24026 hybrid_clock.cc:648] HybridClock initialized: now 1786288457102618 us; error 45 us; skew 500 ppm
I20260809 15:14:17.102974 24026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:17 dist-test-slave-8w9v krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288457, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:17.110046 24026 init.cc:377] Logged in from keytab as oryx/127.15.11.131@KRBTEST.COM (short username oryx)
I20260809 15:14:17.111606 24026 webserver.cc:460] Webserver started at http://127.15.11.131:43873/ using document root <none> and password file <none>
I20260809 15:14:17.112449 24026 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:17.112583 24026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:17.112926 24026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:17.115854 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "bc87148b7e914f9fb35c0e77dbc61fc7"
format_stamp: "Formatted at 2026-08-09 15:14:17 on dist-test-slave-8w9v"
server_key: "96e7f65a8b2d85d8926f6916e73df226"
server_key_iv: "e13a197f5f0e50e91bdfb1ee1cb3fc99"
server_key_version: "encryptionkey@0"
I20260809 15:14:17.116777 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "bc87148b7e914f9fb35c0e77dbc61fc7"
format_stamp: "Formatted at 2026-08-09 15:14:17 on dist-test-slave-8w9v"
server_key: "96e7f65a8b2d85d8926f6916e73df226"
server_key_iv: "e13a197f5f0e50e91bdfb1ee1cb3fc99"
server_key_version: "encryptionkey@0"
I20260809 15:14:17.121811 24026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:14:17.125355 24044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:17.126698 24026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:14:17.126893 24026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "bc87148b7e914f9fb35c0e77dbc61fc7"
format_stamp: "Formatted at 2026-08-09 15:14:17 on dist-test-slave-8w9v"
server_key: "96e7f65a8b2d85d8926f6916e73df226"
server_key_iv: "e13a197f5f0e50e91bdfb1ee1cb3fc99"
server_key_version: "encryptionkey@0"
I20260809 15:14:17.127137 24026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:17.145869 24026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:17.150023 24026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:17.150316 24026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:17.153432 24026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:17.153615 24026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 15:14:17.154004 24026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:17.154145 24026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:17.163194 23868 heartbeater.cc:503] Master 127.15.11.190:38881 was elected leader, sending a full tablet report...
Aug 09 15:14:17 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288457, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:17.176923 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:17.153733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:60803 (local address 127.15.11.190:38881)
0809 15:14:17.153930 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:17.153940 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:17.155311 (+ 1371us) server_negotiation.cc:407] Connection header received
0809 15:14:17.157877 (+ 2566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:17.157883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:17.157977 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:17.158089 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:17.159810 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.160855 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.163730 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.164450 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.164550 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:17.170061 (+ 5511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:17.170068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:17.170073 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:17.170141 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:17.173352 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:17.174391 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:17.174402 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:17.174406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:17.174557 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:17.175017 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:17.175023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:17.175026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:17.175310 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:17.175474 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:17.176317 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 15:14:17.176552 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:14:17.177889 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:17.154292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38881 (local address 127.15.11.131:60803)
0809 15:14:17.155159 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:17.155205 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:17.157584 (+ 2379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:17.158333 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:17.158341 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:17.158872 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:17.159634 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:17.159657 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.161106 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.161117 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:17.163521 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:17.163543 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.164704 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.164712 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:17.165045 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:17.165926 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:17.165971 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:17.169842 (+ 3871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:17.173548 (+ 3706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:17.173555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:17.173570 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:17.174224 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:17.174698 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:17.174701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:17.174703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:17.174900 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:17.175458 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:17.175466 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:17.175871 (+ 405us) client_negotiation.cc:769] Sending connection context
0809 15:14:17.176179 (+ 308us) client_negotiation.cc:240] Negotiation successful
0809 15:14:17.176729 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":393,"threads_started":1}
I20260809 15:14:17.201023 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:17.183795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38881 (local address 127.15.11.131:50467)
0809 15:14:17.184632 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:17.184647 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:17.184813 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:17.185172 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:17.185176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:17.185850 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:17.186221 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:17.186230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.187548 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.187561 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:17.189808 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:17.189829 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.190582 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.190591 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:17.190925 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:17.191752 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:17.191769 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:17.193053 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:17.197024 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:17.197030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:17.197034 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:17.197703 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:17.198164 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:17.198175 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:17.198182 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:17.198386 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:17.199271 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:17.199278 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:17.199591 (+ 313us) client_negotiation.cc:769] Sending connection context
0809 15:14:17.199791 (+ 200us) client_negotiation.cc:240] Negotiation successful
0809 15:14:17.200305 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":491,"threads_started":1}
I20260809 15:14:17.202250 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:17.183778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:50467 (local address 127.15.11.190:38881)
0809 15:14:17.183988 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:17.183996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:17.184740 (+ 744us) server_negotiation.cc:407] Connection header received
0809 15:14:17.184960 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:17.184972 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:17.185038 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:17.185139 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:17.186386 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.187371 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.189971 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.190358 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.190465 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:17.193260 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:17.193264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:17.193268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:17.193297 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:17.196888 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:17.197834 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:17.197839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:17.197843 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:17.198023 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:17.198577 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:17.198586 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:17.198592 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:17.199017 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:17.199200 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:17.201532 (+ 2332us) server_negotiation.cc:299] Negotiation successful
0809 15:14:17.201810 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:14:17.211375 24026 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:37091
I20260809 15:14:17.211400 24176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:37091 every 8 connection(s)
I20260809 15:14:17.212683 24026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:14:17.214437 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 24026
I20260809 15:14:17.214648 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:14:17.214815 24177 heartbeater.cc:347] Connected to a master server at 127.15.11.190:38881
I20260809 15:14:17.215056 24177 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:17.215184 15406 external_mini_cluster.cc:1428] Setting key bccddc70a107aff2b845433ccd17d80c
I20260809 15:14:17.215839 24177 heartbeater.cc:511] Master 127.15.11.190:38881 requested a full tablet report, sending...
I20260809 15:14:17.217365 23643 ts_manager.cc:194] Registered new tserver with Master: bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091)
I20260809 15:14:17.218324 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:14:17.218328 23643 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:60803
Aug 09 15:14:17 dist-test-slave-8w9v krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288457, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:14:17 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:17 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:17.685834 24020 heartbeater.cc:503] Master 127.15.11.190:38881 was elected leader, sending a full tablet report...
I20260809 15:14:18.220714 24177 heartbeater.cc:503] Master 127.15.11.190:38881 was elected leader, sending a full tablet report...
W20260809 15:14:18.341993 24186 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.069s user 0.003s sys 0.014s
W20260809 15:14:18.343292 23748 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:17.267182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.15.11.190:38881)
0809 15:14:17.267415 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:17.267424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:17.267657 (+ 233us) server_negotiation.cc:407] Connection header received
0809 15:14:17.267906 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:17.267911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:17.267993 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:17.268161 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:17.269544 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.270278 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.271610 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:17.272011 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:17.272107 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.343022 (+1070915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47028: BlockingRecv error: recv got EOF from 127.0.0.1:47028 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 15:14:18 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288457, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:18.362344 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.348730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47030 (local address 127.15.11.190:38881)
0809 15:14:18.348970 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:18.348977 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:18.349141 (+ 164us) server_negotiation.cc:407] Connection header received
0809 15:14:18.349294 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:18.349298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:18.349404 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:18.349492 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:18.350914 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.351671 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.353011 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.353436 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.353523 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.357513 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:18.357517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:18.357520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:18.357550 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:18.360053 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:18.360692 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:18.360696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:18.360698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:18.360813 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:18.361123 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:18.361126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:18.361128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:18.361380 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:18.361609 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:18.361791 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 15:14:18.361953 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 15:14:18.366819 23643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:14:18.374068 23643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:14:18.401619 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.387004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:32971 (local address 127.0.0.1:57098)
0809 15:14:18.388045 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:18.388100 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:18.388418 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:18.390178 (+ 1760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:18.390190 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:18.390251 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:18.390879 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.390898 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.393911 (+ 3013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.393920 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.397586 (+ 3666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.397613 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.400392 (+ 2779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.400401 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.400687 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.400780 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 15:14:18.400918 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 15:14:18.401133 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":461,"threads_started":1}
I20260809 15:14:18.403327 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.387308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57098 (local address 127.15.11.129:32971)
0809 15:14:18.389297 (+ 1989us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:18.389310 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:18.389352 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:14:18.389489 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:18.389502 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:18.389833 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:18.391198 (+ 1365us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:18.391503 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.393605 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.397997 (+ 4392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.400088 (+ 2091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.402393 (+ 2305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.402672 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:18.402762 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 15:14:18.402967 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1775,"thread_start_us":1594,"threads_started":1}
I20260809 15:14:18.409541 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.387787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:43443 (local address 127.0.0.1:40234)
0809 15:14:18.400491 (+ 12704us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:18.400521 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:18.400689 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:18.401900 (+ 1211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:18.401905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:18.401936 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:18.402584 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.402599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.404774 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.404783 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.407191 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.407202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.408709 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.408717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.408900 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.408934 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:14:18.409025 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:14:18.409149 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12293,"thread_start_us":12124,"threads_started":1}
I20260809 15:14:18.410354 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.388021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40234 (local address 127.15.11.130:43443)
0809 15:14:18.399657 (+ 11636us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:18.399667 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:18.400861 (+ 1194us) server_negotiation.cc:407] Connection header received
0809 15:14:18.400984 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:18.400993 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:18.401274 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:18.401635 (+ 361us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:18.402813 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.404604 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.407350 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.408484 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.408640 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.408852 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:18.409877 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0809 15:14:18.410057 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11404,"thread_start_us":12097,"threads_started":1}
I20260809 15:14:18.409776 23794 tablet_service.cc:1467] Processing CreateTablet for tablet 923b689e2d804173aaf8b746d2f6d1e9 (DEFAULT_TABLE table=test-table [id=53786c516c7d4afc9da8415c0fbe8b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:18.411521 23794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923b689e2d804173aaf8b746d2f6d1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:18.411543 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.389304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:37091 (local address 127.0.0.1:60388)
0809 15:14:18.401922 (+ 12618us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:18.401939 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:18.402056 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:18.404833 (+ 2777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:18.404841 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:18.404885 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:18.405227 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.405239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.406901 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.406907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.408849 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.408867 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.410775 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.410780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.410964 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.410995 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 15:14:18.411118 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 15:14:18.411251 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12525,"thread_start_us":15659,"threads_started":1}
I20260809 15:14:18.411720 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.389337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.15.11.131:37091)
0809 15:14:18.404152 (+ 14815us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:18.404163 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:18.404194 (+ 31us) server_negotiation.cc:407] Connection header received
0809 15:14:18.404309 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:18.404318 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:18.404573 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:18.405179 (+ 606us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:18.405462 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.406674 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.409063 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.410572 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.410755 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.410968 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:18.411188 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 15:14:18.411373 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14599,"thread_start_us":14460,"threads_started":1}
I20260809 15:14:18.414139 23942 tablet_service.cc:1467] Processing CreateTablet for tablet 923b689e2d804173aaf8b746d2f6d1e9 (DEFAULT_TABLE table=test-table [id=53786c516c7d4afc9da8415c0fbe8b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:18.415256 23942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923b689e2d804173aaf8b746d2f6d1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:18.414992 24104 tablet_service.cc:1467] Processing CreateTablet for tablet 923b689e2d804173aaf8b746d2f6d1e9 (DEFAULT_TABLE table=test-table [id=53786c516c7d4afc9da8415c0fbe8b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:18.416226 24104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 923b689e2d804173aaf8b746d2f6d1e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:18.429082 24206 tablet_bootstrap.cc:492] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: Bootstrap starting.
I20260809 15:14:18.430303 24207 tablet_bootstrap.cc:492] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Bootstrap starting.
I20260809 15:14:18.431586 24208 tablet_bootstrap.cc:492] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: Bootstrap starting.
I20260809 15:14:18.433995 24206 tablet_bootstrap.cc:654] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:18.434514 24208 tablet_bootstrap.cc:654] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:18.434572 24207 tablet_bootstrap.cc:654] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:18.435523 24208 log.cc:824] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:18.435488 24206 log.cc:824] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:18.437114 24207 log.cc:824] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:18.438859 24208 tablet_bootstrap.cc:492] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: No bootstrap required, opened a new log
I20260809 15:14:18.439162 24208 ts_tablet_manager.cc:1397] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 15:14:18.440244 24207 tablet_bootstrap.cc:492] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: No bootstrap required, opened a new log
I20260809 15:14:18.440310 24206 tablet_bootstrap.cc:492] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: No bootstrap required, opened a new log
I20260809 15:14:18.440562 24207 ts_tablet_manager.cc:1397] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260809 15:14:18.440580 24206 ts_tablet_manager.cc:1397] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260809 15:14:18.448859 24207 raft_consensus.cc:348] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } }
I20260809 15:14:18.449249 24207 raft_consensus.cc:374] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:18.449390 24207 raft_consensus.cc:729] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ae7eb3a8994ddb8ae88c9c32e428f9, State: Initialized, Role: FOLLOWER
I20260809 15:14:18.449735 24208 raft_consensus.cc:348] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } }
I20260809 15:14:18.450140 24208 raft_consensus.cc:374] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:18.450021 24207 consensus_queue.cc:260] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } }
I20260809 15:14:18.450268 24208 raft_consensus.cc:729] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e56413686b4d6a9b2c04dab129457d, State: Initialized, Role: FOLLOWER
I20260809 15:14:18.450863 24208 consensus_queue.cc:260] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } }
I20260809 15:14:18.452054 24207 ts_tablet_manager.cc:1428] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260809 15:14:18.451035 24206 raft_consensus.cc:348] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } }
I20260809 15:14:18.452953 24206 raft_consensus.cc:374] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:18.453121 24206 raft_consensus.cc:729] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc87148b7e914f9fb35c0e77dbc61fc7, State: Initialized, Role: FOLLOWER
I20260809 15:14:18.453305 24208 ts_tablet_manager.cc:1428] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260809 15:14:18.453835 24206 consensus_queue.cc:260] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } }
I20260809 15:14:18.455832 24206 ts_tablet_manager.cc:1428] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent starting tablet: real 0.015s user 0.007s sys 0.007s
W20260809 15:14:18.465955 24179 tablet.cc:2369] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:14:18.506848 24214 raft_consensus.cc:482] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:18.507292 24214 raft_consensus.cc:504] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } }
I20260809 15:14:18.509080 24214 leader_election.cc:288] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971), 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443)
I20260809 15:14:18.516196 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.509653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:49659 (local address 127.15.11.129:32971)
0809 15:14:18.509829 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:18.509835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:18.510461 (+ 626us) server_negotiation.cc:407] Connection header received
0809 15:14:18.510721 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:18.510725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:18.510813 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:18.510935 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:18.511599 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.512535 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.514463 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.515260 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.515435 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.515511 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:18.515766 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 15:14:18.515912 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:14:18.516314 24215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.509553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:32971 (local address 127.15.11.131:49659)
0809 15:14:18.510303 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:18.510329 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:18.510543 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:18.510971 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:18.510974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:18.511018 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:18.511426 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.511440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.512665 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.512670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.514309 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.514321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.515464 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.515469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.515610 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.515632 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:14:18.515734 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 15:14:18.515867 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":60,"thread_start_us":1035,"threads_started":1}
I20260809 15:14:18.516778 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.509616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:56521 (local address 127.15.11.130:43443)
0809 15:14:18.509830 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:18.509835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:18.511028 (+ 1193us) server_negotiation.cc:407] Connection header received
0809 15:14:18.511202 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:18.511206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:18.511314 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:18.511421 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:18.512012 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.512796 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.514571 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.515665 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.515797 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.515878 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:18.516434 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 15:14:18.516560 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:14:18.517408 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.509569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:43443 (local address 127.15.11.131:56521)
0809 15:14:18.510924 (+ 1355us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:18.510941 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:18.511056 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:18.511440 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:18.511443 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:18.511461 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:18.511861 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.511874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.512945 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.512949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.514398 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.514416 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.516063 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.516069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.516260 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.516292 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:14:18.516957 (+ 665us) client_negotiation.cc:240] Negotiation successful
0809 15:14:18.517078 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1231,"mutex_wait_us":1,"thread_start_us":373,"threads_started":1}
I20260809 15:14:18.517360 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923b689e2d804173aaf8b746d2f6d1e9" candidate_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e56413686b4d6a9b2c04dab129457d" is_pre_election: true
I20260809 15:14:18.517915 23815 raft_consensus.cc:2454] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc87148b7e914f9fb35c0e77dbc61fc7 in term 0.
I20260809 15:14:18.518648 24047 leader_election.cc:302] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc87148b7e914f9fb35c0e77dbc61fc7, f0e56413686b4d6a9b2c04dab129457d; no voters:
I20260809 15:14:18.518584 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923b689e2d804173aaf8b746d2f6d1e9" candidate_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" is_pre_election: true
I20260809 15:14:18.519096 24214 raft_consensus.cc:2781] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:14:18.519111 23962 raft_consensus.cc:2454] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc87148b7e914f9fb35c0e77dbc61fc7 in term 0.
I20260809 15:14:18.519239 24214 raft_consensus.cc:482] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:14:18.519361 24214 raft_consensus.cc:3037] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:18.522507 24214 raft_consensus.cc:504] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } }
I20260809 15:14:18.523197 24214 leader_election.cc:288] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [CANDIDATE]: Term 1 election: Requested vote from peers f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971), 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443)
I20260809 15:14:18.523639 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923b689e2d804173aaf8b746d2f6d1e9" candidate_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e56413686b4d6a9b2c04dab129457d"
I20260809 15:14:18.523685 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "923b689e2d804173aaf8b746d2f6d1e9" candidate_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9"
I20260809 15:14:18.523841 23815 raft_consensus.cc:3037] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:18.523912 23962 raft_consensus.cc:3037] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:18.526158 23962 raft_consensus.cc:2454] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc87148b7e914f9fb35c0e77dbc61fc7 in term 1.
I20260809 15:14:18.526607 24045 leader_election.cc:302] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ae7eb3a8994ddb8ae88c9c32e428f9, bc87148b7e914f9fb35c0e77dbc61fc7; no voters:
I20260809 15:14:18.526923 24214 raft_consensus.cc:2781] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:18.526955 23815 raft_consensus.cc:2454] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc87148b7e914f9fb35c0e77dbc61fc7 in term 1.
I20260809 15:14:18.527551 24214 raft_consensus.cc:686] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 LEADER]: Becoming Leader. State: Replica: bc87148b7e914f9fb35c0e77dbc61fc7, State: Running, Role: LEADER
I20260809 15:14:18.528100 24214 consensus_queue.cc:237] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } }
I20260809 15:14:18.533317 23643 catalog_manager.cc:5495] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131). New cstate: current_term: 1 leader_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } health_report { overall_health: UNKNOWN } } }
I20260809 15:14:18.572243 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.566894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.15.11.131:37091)
0809 15:14:18.567099 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:18.567106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:18.567146 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:14:18.567455 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:18.567459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:18.567560 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:18.567707 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:14:18.568106 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.568808 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.570070 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.570481 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.570612 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.571074 (+ 462us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:14:18.571605 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:14:18.571766 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:18.571818 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 15:14:18.571895 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:14:18.580858 23815 raft_consensus.cc:1261] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Refusing update from remote peer bc87148b7e914f9fb35c0e77dbc61fc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:18.580916 23962 raft_consensus.cc:1261] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Refusing update from remote peer bc87148b7e914f9fb35c0e77dbc61fc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:18.581902 24217 consensus_queue.cc:1035] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:18.582261 24214 consensus_queue.cc:1035] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:18.592720 24222 mvcc.cc:205] Tried to move back new op lower bound from 7316637526336098304 to 7316637526136258560. Current Snapshot: MvccSnapshot[applied={T|T < 7316637526336098304}]
I20260809 15:14:18.611132 23643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:18.611403 23643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:18.615020 23643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf: Sending DeleteTablet for 3 replicas of tablet 923b689e2d804173aaf8b746d2f6d1e9
I20260809 15:14:18.616299 24104 tablet_service.cc:1514] Processing DeleteTablet for tablet 923b689e2d804173aaf8b746d2f6d1e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:18 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:60388
I20260809 15:14:18.616393 23794 tablet_service.cc:1514] Processing DeleteTablet for tablet 923b689e2d804173aaf8b746d2f6d1e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:18 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:57098
I20260809 15:14:18.616585 23942 tablet_service.cc:1514] Processing DeleteTablet for tablet 923b689e2d804173aaf8b746d2f6d1e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:18 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:40234
I20260809 15:14:18.617548 24233 tablet_replica.cc:330] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: stopping tablet replica
I20260809 15:14:18.618060 24233 raft_consensus.cc:2229] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:18.618664 24233 raft_consensus.cc:2258] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:18.618474 23643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47030:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:14:18.619177 23643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:14:18.620931 24230 tablet_replica.cc:330] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: stopping tablet replica
I20260809 15:14:18.621330 24230 raft_consensus.cc:2229] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:14:18.621408 24233 ts_tablet_manager.cc:1905] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:18.622112 24230 raft_consensus.cc:2258] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:18.622850 24232 tablet_replica.cc:330] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: stopping tablet replica
I20260809 15:14:18.623250 24232 raft_consensus.cc:2229] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:18.623699 24232 raft_consensus.cc:2258] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:18.625474 24230 ts_tablet_manager.cc:1905] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:18.629027 24233 ts_tablet_manager.cc:1918] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:18.629271 24233 log.cc:1196] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/923b689e2d804173aaf8b746d2f6d1e9
I20260809 15:14:18.629468 24232 ts_tablet_manager.cc:1905] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:18.630014 24233 ts_tablet_manager.cc:1939] T 923b689e2d804173aaf8b746d2f6d1e9 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting consensus metadata
I20260809 15:14:18.629920 23942 tablet_service.cc:1467] Processing CreateTablet for tablet 0941a77b1bf44546b255834c4ba101d2 (DEFAULT_TABLE table=test-table [id=3ac441c55ada499fb5155b4555c3b703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:18.630477 23942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0941a77b1bf44546b255834c4ba101d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:18.630813 24104 tablet_service.cc:1467] Processing CreateTablet for tablet 0941a77b1bf44546b255834c4ba101d2 (DEFAULT_TABLE table=test-table [id=3ac441c55ada499fb5155b4555c3b703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:18.630836 23793 tablet_service.cc:1467] Processing CreateTablet for tablet 0941a77b1bf44546b255834c4ba101d2 (DEFAULT_TABLE table=test-table [id=3ac441c55ada499fb5155b4555c3b703]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:18.631247 24104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0941a77b1bf44546b255834c4ba101d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:18.631285 23793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0941a77b1bf44546b255834c4ba101d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:18.636596 24207 tablet_bootstrap.cc:492] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Bootstrap starting.
I20260809 15:14:18.637724 24206 tablet_bootstrap.cc:492] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: Bootstrap starting.
I20260809 15:14:18.637807 24208 tablet_bootstrap.cc:492] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: Bootstrap starting.
I20260809 15:14:18.638166 24207 tablet_bootstrap.cc:654] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:18.639372 24206 tablet_bootstrap.cc:654] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:18.639497 24208 tablet_bootstrap.cc:654] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:18.647368 23627 catalog_manager.cc:4841] TS 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443): tablet 923b689e2d804173aaf8b746d2f6d1e9 (table test-table [id=53786c516c7d4afc9da8415c0fbe8b95]) successfully deleted
I20260809 15:14:18.649392 24207 tablet_bootstrap.cc:492] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: No bootstrap required, opened a new log
I20260809 15:14:18.649542 24207 ts_tablet_manager.cc:1397] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.003s
I20260809 15:14:18.650107 24208 tablet_bootstrap.cc:492] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: No bootstrap required, opened a new log
I20260809 15:14:18.650322 24208 ts_tablet_manager.cc:1397] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260809 15:14:18.650411 24207 raft_consensus.cc:348] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:18.650650 24207 raft_consensus.cc:374] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:18.650766 24207 raft_consensus.cc:729] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ae7eb3a8994ddb8ae88c9c32e428f9, State: Initialized, Role: FOLLOWER
I20260809 15:14:18.651046 24207 consensus_queue.cc:260] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:18.651604 24208 raft_consensus.cc:348] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:18.651872 24208 raft_consensus.cc:374] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:18.652045 24208 raft_consensus.cc:729] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e56413686b4d6a9b2c04dab129457d, State: Initialized, Role: FOLLOWER
I20260809 15:14:18.652202 24230 ts_tablet_manager.cc:1918] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:18.652468 24230 log.cc:1196] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/923b689e2d804173aaf8b746d2f6d1e9
I20260809 15:14:18.652974 24208 consensus_queue.cc:260] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:18.653358 24206 tablet_bootstrap.cc:492] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: No bootstrap required, opened a new log
I20260809 15:14:18.653540 24206 ts_tablet_manager.cc:1397] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.000s
I20260809 15:14:18.654214 24208 ts_tablet_manager.cc:1428] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 15:14:18.654454 24206 raft_consensus.cc:348] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:18.654691 24206 raft_consensus.cc:374] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:18.654785 24206 raft_consensus.cc:729] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc87148b7e914f9fb35c0e77dbc61fc7, State: Initialized, Role: FOLLOWER
I20260809 15:14:18.655058 24206 consensus_queue.cc:260] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:18.655335 24207 ts_tablet_manager.cc:1428] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260809 15:14:18.656262 24206 ts_tablet_manager.cc:1428] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:14:18.656996 24230 ts_tablet_manager.cc:1939] T 923b689e2d804173aaf8b746d2f6d1e9 P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting consensus metadata
W20260809 15:14:18.658208 23869 tablet.cc:2369] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:14:18.659237 23629 catalog_manager.cc:4841] TS bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091): tablet 923b689e2d804173aaf8b746d2f6d1e9 (table test-table [id=53786c516c7d4afc9da8415c0fbe8b95]) successfully deleted
I20260809 15:14:18.662345 24232 ts_tablet_manager.cc:1918] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:18.662510 24232 log.cc:1196] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/923b689e2d804173aaf8b746d2f6d1e9
I20260809 15:14:18.663125 24232 ts_tablet_manager.cc:1939] T 923b689e2d804173aaf8b746d2f6d1e9 P f0e56413686b4d6a9b2c04dab129457d: Deleting consensus metadata
I20260809 15:14:18.664880 23629 catalog_manager.cc:4841] TS f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971): tablet 923b689e2d804173aaf8b746d2f6d1e9 (table test-table [id=53786c516c7d4afc9da8415c0fbe8b95]) successfully deleted
W20260809 15:14:18.681238 24022 tablet.cc:2369] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:14:18.938030 24213 raft_consensus.cc:482] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:18.938308 24213 raft_consensus.cc:504] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:18.939975 24213 leader_election.cc:288] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443), bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091)
I20260809 15:14:18.948729 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.940532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:44889 (local address 127.15.11.131:37091)
0809 15:14:18.940790 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:18.940799 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:18.941688 (+ 889us) server_negotiation.cc:407] Connection header received
0809 15:14:18.941951 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:18.941977 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:18.942059 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:18.942169 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:18.942950 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.943984 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.946182 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.947529 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.947733 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.947825 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:18.948248 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 15:14:18.948410 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:14:18.948761 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.940459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:37091 (local address 127.15.11.129:44889)
0809 15:14:18.941599 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:18.941616 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:18.941779 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:18.942244 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:18.942249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:18.942316 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:18.942753 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.942768 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.944106 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.944112 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.946006 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.946019 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.947795 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.947802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.947986 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.948037 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 15:14:18.948210 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 15:14:18.948371 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":89,"thread_start_us":521,"threads_started":1}
I20260809 15:14:18.948843 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.940745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:52587 (local address 127.15.11.130:43443)
0809 15:14:18.940945 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:18.940951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:18.941449 (+ 498us) server_negotiation.cc:407] Connection header received
0809 15:14:18.941771 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:18.941777 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:18.941887 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:18.942000 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:18.942629 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.943673 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.946426 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.947696 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.947842 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.947920 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:18.948450 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 15:14:18.948584 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:14:18.949450 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:18.940338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:43443 (local address 127.15.11.129:52587)
0809 15:14:18.941323 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:18.941346 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:18.941570 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:18.942024 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:18.942029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:18.942055 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:18.942471 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.942482 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.943829 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.943837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.946236 (+ 2399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:18.946251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:18.948050 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:18.948054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:18.948274 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:18.948305 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 15:14:18.948941 (+ 636us) client_negotiation.cc:240] Negotiation successful
0809 15:14:18.949068 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"mutex_wait_us":22,"thread_start_us":985,"threads_started":1}
I20260809 15:14:18.950078 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0941a77b1bf44546b255834c4ba101d2" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" is_pre_election: true
I20260809 15:14:18.949994 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0941a77b1bf44546b255834c4ba101d2" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" is_pre_election: true
I20260809 15:14:18.950354 23962 raft_consensus.cc:2454] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 0.
I20260809 15:14:18.950469 24127 raft_consensus.cc:2454] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 0.
I20260809 15:14:18.950899 23740 leader_election.cc:302] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ae7eb3a8994ddb8ae88c9c32e428f9, f0e56413686b4d6a9b2c04dab129457d; no voters:
I20260809 15:14:18.951284 24213 raft_consensus.cc:2781] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:14:18.951423 24213 raft_consensus.cc:482] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:14:18.951556 24213 raft_consensus.cc:3037] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:18.953845 24213 raft_consensus.cc:504] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:18.954448 24213 leader_election.cc:288] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 election: Requested vote from peers 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443), bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091)
I20260809 15:14:18.954859 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0941a77b1bf44546b255834c4ba101d2" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9"
I20260809 15:14:18.954903 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0941a77b1bf44546b255834c4ba101d2" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7"
I20260809 15:14:18.955062 23962 raft_consensus.cc:3037] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:18.955116 24127 raft_consensus.cc:3037] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:18.957273 23962 raft_consensus.cc:2454] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 1.
I20260809 15:14:18.957621 23740 leader_election.cc:302] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ae7eb3a8994ddb8ae88c9c32e428f9, f0e56413686b4d6a9b2c04dab129457d; no voters:
I20260809 15:14:18.957872 24213 raft_consensus.cc:2781] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:18.958356 24213 raft_consensus.cc:686] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 1 LEADER]: Becoming Leader. State: Replica: f0e56413686b4d6a9b2c04dab129457d, State: Running, Role: LEADER
I20260809 15:14:18.958566 24127 raft_consensus.cc:2454] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 1.
I20260809 15:14:18.959076 24213 consensus_queue.cc:237] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:18.962890 23643 catalog_manager.cc:5495] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d reported cstate change: term changed from 0 to 1, leader changed from <none> to f0e56413686b4d6a9b2c04dab129457d (127.15.11.129). New cstate: current_term: 1 leader_uuid: "f0e56413686b4d6a9b2c04dab129457d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } health_report { overall_health: UNKNOWN } } }
Aug 09 15:14:19 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288455, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.190@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:19.038398 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.022503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:38881 (local address 127.0.0.1:47040)
0809 15:14:19.023228 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:19.023290 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:19.023473 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:19.023869 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:19.023879 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:19.024840 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:19.025199 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:19.025212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.026311 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.026318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:19.028101 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:19.028147 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.028866 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.028874 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:19.029074 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.029859 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:19.029887 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:19.032782 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:19.035760 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:19.035774 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:19.035781 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:19.036315 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:19.036749 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:19.036761 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:19.036763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:19.036895 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:19.037416 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:19.037424 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:19.037619 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 15:14:19.037809 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 15:14:19.038085 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":342,"threads_started":1}
I20260809 15:14:19.038771 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.022526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.15.11.190:38881)
0809 15:14:19.023572 (+ 1046us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:19.023576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:19.023597 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:14:19.023642 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:19.023646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:19.023703 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:19.023792 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:19.025374 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.026171 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.028289 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.028693 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.028783 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.033014 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:19.033019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:19.033022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:19.033049 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:19.035628 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:19.036483 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:19.036488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:19.036494 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:19.036629 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:19.037009 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:19.037014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:19.037017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:19.037290 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:19.037456 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:19.037975 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 15:14:19.038283 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":933,"thread_start_us":756,"threads_started":1}
I20260809 15:14:19.043905 23643 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:47040:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 15:14:19.045130 23643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:14:19.052059 23793 tablet_service.cc:1467] Processing CreateTablet for tablet e51573fab7434f84abf1c1da5fb72789 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8508c9e826e74f2581d3deeb434ca9cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:14:19.052158 23942 tablet_service.cc:1467] Processing CreateTablet for tablet e51573fab7434f84abf1c1da5fb72789 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8508c9e826e74f2581d3deeb434ca9cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:14:19.052733 23942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51573fab7434f84abf1c1da5fb72789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:19.052739 23793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51573fab7434f84abf1c1da5fb72789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:19.057191 24104 tablet_service.cc:1467] Processing CreateTablet for tablet e51573fab7434f84abf1c1da5fb72789 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8508c9e826e74f2581d3deeb434ca9cc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:14:19.057865 24104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e51573fab7434f84abf1c1da5fb72789. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:19.058599 24207 tablet_bootstrap.cc:492] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Bootstrap starting.
I20260809 15:14:19.059785 24208 tablet_bootstrap.cc:492] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: Bootstrap starting.
I20260809 15:14:19.060170 24207 tablet_bootstrap.cc:654] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:19.061750 24208 tablet_bootstrap.cc:654] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:19.063640 24207 tablet_bootstrap.cc:492] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: No bootstrap required, opened a new log
I20260809 15:14:19.063884 24207 ts_tablet_manager.cc:1397] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260809 15:14:19.064639 24208 tablet_bootstrap.cc:492] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: No bootstrap required, opened a new log
I20260809 15:14:19.064815 24208 ts_tablet_manager.cc:1397] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260809 15:14:19.064852 24206 tablet_bootstrap.cc:492] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: Bootstrap starting.
I20260809 15:14:19.064744 24207 raft_consensus.cc:348] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.065027 24207 raft_consensus.cc:374] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:19.065119 24207 raft_consensus.cc:729] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ae7eb3a8994ddb8ae88c9c32e428f9, State: Initialized, Role: FOLLOWER
I20260809 15:14:19.065419 24207 consensus_queue.cc:260] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.066148 24207 ts_tablet_manager.cc:1428] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 15:14:19.065980 24212 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.066421 24212 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:14:19.066751 24206 tablet_bootstrap.cc:654] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:19.065605 24208 raft_consensus.cc:348] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.067279 24208 raft_consensus.cc:374] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:19.067396 24208 raft_consensus.cc:729] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e56413686b4d6a9b2c04dab129457d, State: Initialized, Role: FOLLOWER
I20260809 15:14:19.067653 24208 consensus_queue.cc:260] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.068439 24208 ts_tablet_manager.cc:1428] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 15:14:19.068272 24213 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.069155 24213 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:14:19.069347 24206 tablet_bootstrap.cc:492] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: No bootstrap required, opened a new log
I20260809 15:14:19.069527 24206 ts_tablet_manager.cc:1397] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:14:19.070163 24206 raft_consensus.cc:348] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.070465 24206 raft_consensus.cc:374] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:19.070569 24206 raft_consensus.cc:729] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc87148b7e914f9fb35c0e77dbc61fc7, State: Initialized, Role: FOLLOWER
I20260809 15:14:19.070844 24206 consensus_queue.cc:260] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.071458 24206 ts_tablet_manager.cc:1428] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 15:14:19.071336 24217 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.071636 24217 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:14:19.087071 24217 raft_consensus.cc:482] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:19.087260 24217 raft_consensus.cc:504] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.087934 24217 leader_election.cc:288] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971), 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443)
I20260809 15:14:19.088465 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51573fab7434f84abf1c1da5fb72789" candidate_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e56413686b4d6a9b2c04dab129457d" is_pre_election: true
I20260809 15:14:19.088572 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51573fab7434f84abf1c1da5fb72789" candidate_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" is_pre_election: true
I20260809 15:14:19.088749 23815 raft_consensus.cc:2454] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc87148b7e914f9fb35c0e77dbc61fc7 in term 0.
I20260809 15:14:19.088850 23962 raft_consensus.cc:2454] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc87148b7e914f9fb35c0e77dbc61fc7 in term 0.
I20260809 15:14:19.089242 24047 leader_election.cc:302] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 44ae7eb3a8994ddb8ae88c9c32e428f9, bc87148b7e914f9fb35c0e77dbc61fc7, f0e56413686b4d6a9b2c04dab129457d; no voters:
I20260809 15:14:19.089563 24217 raft_consensus.cc:2781] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:14:19.089701 24217 raft_consensus.cc:482] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:14:19.089812 24217 raft_consensus.cc:3037] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:19.092074 24217 raft_consensus.cc:504] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.092638 24217 leader_election.cc:288] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [CANDIDATE]: Term 1 election: Requested vote from peers f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971), 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443)
I20260809 15:14:19.093143 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51573fab7434f84abf1c1da5fb72789" candidate_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0e56413686b4d6a9b2c04dab129457d"
I20260809 15:14:19.093123 23962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e51573fab7434f84abf1c1da5fb72789" candidate_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9"
I20260809 15:14:19.093343 23815 raft_consensus.cc:3037] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:19.093382 23962 raft_consensus.cc:3037] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:19.096364 23815 raft_consensus.cc:2454] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc87148b7e914f9fb35c0e77dbc61fc7 in term 1.
I20260809 15:14:19.096351 23962 raft_consensus.cc:2454] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc87148b7e914f9fb35c0e77dbc61fc7 in term 1.
I20260809 15:14:19.096763 24047 leader_election.cc:302] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc87148b7e914f9fb35c0e77dbc61fc7, f0e56413686b4d6a9b2c04dab129457d; no voters:
I20260809 15:14:19.097200 24217 raft_consensus.cc:2781] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:19.097404 24217 raft_consensus.cc:686] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 LEADER]: Becoming Leader. State: Replica: bc87148b7e914f9fb35c0e77dbc61fc7, State: Running, Role: LEADER
I20260809 15:14:19.097673 24217 consensus_queue.cc:237] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.099033 24228 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: TxnStatusTablet state changed. Reason: New leader bc87148b7e914f9fb35c0e77dbc61fc7. Latest consensus state: current_term: 1 leader_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.099346 24228 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:14:19.100538 24254 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 15:14:19.100777 24254 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 15:14:19.100708 23643 catalog_manager.cc:5495] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131). New cstate: current_term: 1 leader_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } health_report { overall_health: HEALTHY } } }
I20260809 15:14:19.120507 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.114850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.15.11.131:37091)
0809 15:14:19.115040 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:19.115047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:19.115256 (+ 209us) server_negotiation.cc:407] Connection header received
0809 15:14:19.115505 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:19.115508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:19.115566 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:19.115637 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:14:19.116055 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.116889 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.118811 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.119232 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.119321 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.119857 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:14:19.119956 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:14:19.120019 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:19.120225 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 15:14:19.120287 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:14:19.120661 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.114918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:37091 (local address 127.0.0.1:60402)
0809 15:14:19.115164 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:19.115184 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:19.115329 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:19.115695 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:19.115700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:19.115712 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:14:19.115888 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:19.115899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.117029 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.117035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:19.118665 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:19.118681 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.119402 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.119411 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:19.119603 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.119674 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:14:19.120053 (+ 379us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:14:19.120074 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 15:14:19.120183 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 15:14:19.120340 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 15:14:19.124416 23815 raft_consensus.cc:1261] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Refusing update from remote peer bc87148b7e914f9fb35c0e77dbc61fc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:19.124500 23962 raft_consensus.cc:1261] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Refusing update from remote peer bc87148b7e914f9fb35c0e77dbc61fc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:19.125038 24228 consensus_queue.cc:1035] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:19.125378 24217 consensus_queue.cc:1035] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:19.137019 24213 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: TxnStatusTablet state changed. Reason: New leader bc87148b7e914f9fb35c0e77dbc61fc7. Latest consensus state: current_term: 1 leader_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.137281 24213 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:14:19.139068 24257 mvcc.cc:205] Tried to move back new op lower bound from 7316637528567447552 to 7316637528465960960. Current Snapshot: MvccSnapshot[applied={T|T < 7316637528567447552}]
I20260809 15:14:19.139462 24217 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.139772 24217 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:14:19.140589 24258 mvcc.cc:205] Tried to move back new op lower bound from 7316637528567447552 to 7316637528465960960. Current Snapshot: MvccSnapshot[applied={T|T < 7316637528567447552}]
I20260809 15:14:19.142997 24213 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.143313 24213 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P f0e56413686b4d6a9b2c04dab129457d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:14:19.147519 24212 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: TxnStatusTablet state changed. Reason: New leader bc87148b7e914f9fb35c0e77dbc61fc7. Latest consensus state: current_term: 1 leader_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.147835 24212 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:14:19.148499 24228 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.148854 24228 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P bc87148b7e914f9fb35c0e77dbc61fc7: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:14:19.156342 24212 tablet_replica.cc:439] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } } }
I20260809 15:14:19.156629 24212 tablet_replica.cc:442] T e51573fab7434f84abf1c1da5fb72789 P 44ae7eb3a8994ddb8ae88c9c32e428f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:14:19.162806 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.151812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57108 (local address 127.15.11.129:32971)
0809 15:14:19.154548 (+ 2736us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:19.154557 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:19.155156 (+ 599us) server_negotiation.cc:407] Connection header received
0809 15:14:19.155412 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:19.155417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:19.155525 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:19.155703 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:14:19.156747 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.157966 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.159925 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.160587 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.160705 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.161227 (+ 522us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:14:19.161975 (+ 748us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:14:19.162113 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:19.162249 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 15:14:19.162331 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2553,"thread_start_us":2386,"threads_started":1}
I20260809 15:14:19.186456 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.180281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:37475 (local address 127.15.11.131:37091)
0809 15:14:19.180453 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:19.180457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:19.180973 (+ 516us) server_negotiation.cc:407] Connection header received
0809 15:14:19.181176 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:19.181180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:19.181241 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:19.181330 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:14:19.181822 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.182720 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.184748 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.185174 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.185268 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.185771 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:14:19.185871 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:14:19.185939 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:19.186142 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 15:14:19.186221 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:14:19.186658 24272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.180283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:37091 (local address 127.15.11.129:37475)
0809 15:14:19.180847 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:19.180886 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:19.181037 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:19.181344 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:19.181348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:19.181359 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:14:19.181668 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:19.181676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.182890 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.182896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:19.184603 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:19.184619 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.185345 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.185351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:19.185540 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.185608 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:14:19.185972 (+ 364us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:14:19.185991 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 15:14:19.186118 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 15:14:19.186278 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":307,"threads_started":1}
I20260809 15:14:19.195839 23961 raft_consensus.cc:1261] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Refusing update from remote peer f0e56413686b4d6a9b2c04dab129457d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:19.196403 24127 raft_consensus.cc:1261] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Refusing update from remote peer f0e56413686b4d6a9b2c04dab129457d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:19.196602 24239 consensus_queue.cc:1035] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:19.197073 24213 consensus_queue.cc:1035] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:19.260082 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.243935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:49483 (local address 127.15.11.129:32971)
0809 15:14:19.244109 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:19.244114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:19.252833 (+ 8719us) server_negotiation.cc:407] Connection header received
0809 15:14:19.253176 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:19.253183 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:19.253276 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:19.253346 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:14:19.254285 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.255421 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.257585 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.258241 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.258353 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.258951 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:14:19.259091 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:14:19.259156 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:19.259691 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 15:14:19.259793 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:14:19.260836 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.243508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:32971 (local address 127.15.11.131:49483)
0809 15:14:19.252720 (+ 9212us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:19.252748 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:19.252968 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:19.253591 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:19.253595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:19.253606 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:14:19.254059 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:19.254075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.255554 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.255560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:19.257420 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:19.257436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.258485 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.258491 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:19.258677 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.258746 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:14:19.259562 (+ 816us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:14:19.259571 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:14:19.260307 (+ 736us) client_negotiation.cc:240] Negotiation successful
0809 15:14:19.260446 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8964,"thread_start_us":9787,"threads_started":1}
I20260809 15:14:19.301508 23642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:19.301775 23642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47030:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:19.305519 23642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf: Sending DeleteTablet for 3 replicas of tablet 0941a77b1bf44546b255834c4ba101d2
I20260809 15:14:19.306282 23793 tablet_service.cc:1514] Processing DeleteTablet for tablet 0941a77b1bf44546b255834c4ba101d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:19 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:57098
I20260809 15:14:19.306731 23942 tablet_service.cc:1514] Processing DeleteTablet for tablet 0941a77b1bf44546b255834c4ba101d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:19 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:40234
I20260809 15:14:19.307135 24104 tablet_service.cc:1514] Processing DeleteTablet for tablet 0941a77b1bf44546b255834c4ba101d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:19 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:60388
I20260809 15:14:19.331398 24282 tablet_replica.cc:330] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: stopping tablet replica
I20260809 15:14:19.331642 24282 raft_consensus.cc:2229] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:19.331954 24282 raft_consensus.cc:2258] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:19.332428 24283 tablet_replica.cc:330] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: stopping tablet replica
I20260809 15:14:19.332638 24281 tablet_replica.cc:330] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: stopping tablet replica
I20260809 15:14:19.332742 24283 raft_consensus.cc:2229] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:19.332854 24281 raft_consensus.cc:2229] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:14:19.333114 24283 raft_consensus.cc:2258] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:19.333473 24281 raft_consensus.cc:2258] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:19.334491 24283 ts_tablet_manager.cc:1905] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:19.335614 24282 ts_tablet_manager.cc:1905] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:19.341043 24281 ts_tablet_manager.cc:1905] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:19.342144 24283 ts_tablet_manager.cc:1918] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:14:19.342324 24283 log.cc:1196] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/0941a77b1bf44546b255834c4ba101d2
I20260809 15:14:19.342891 24283 ts_tablet_manager.cc:1939] T 0941a77b1bf44546b255834c4ba101d2 P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting consensus metadata
I20260809 15:14:19.344092 23629 catalog_manager.cc:4841] TS bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091): tablet 0941a77b1bf44546b255834c4ba101d2 (table test-table [id=3ac441c55ada499fb5155b4555c3b703]) successfully deleted
I20260809 15:14:19.349515 24282 ts_tablet_manager.cc:1918] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:14:19.352914 24281 ts_tablet_manager.cc:1918] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:14:19.353132 24282 log.cc:1196] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/0941a77b1bf44546b255834c4ba101d2
I20260809 15:14:19.353704 24282 ts_tablet_manager.cc:1939] T 0941a77b1bf44546b255834c4ba101d2 P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting consensus metadata
I20260809 15:14:19.354912 23627 catalog_manager.cc:4841] TS 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443): tablet 0941a77b1bf44546b255834c4ba101d2 (table test-table [id=3ac441c55ada499fb5155b4555c3b703]) successfully deleted
I20260809 15:14:19.355247 24281 log.cc:1196] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/0941a77b1bf44546b255834c4ba101d2
I20260809 15:14:19.355799 24281 ts_tablet_manager.cc:1939] T 0941a77b1bf44546b255834c4ba101d2 P f0e56413686b4d6a9b2c04dab129457d: Deleting consensus metadata
I20260809 15:14:19.357163 23629 catalog_manager.cc:4841] TS f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971): tablet 0941a77b1bf44546b255834c4ba101d2 (table test-table [id=3ac441c55ada499fb5155b4555c3b703]) successfully deleted
Aug 09 15:14:19 dist-test-slave-8w9v krb5kdc[23594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:14:19 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:19 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:14:19.411062 24249 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:19.396706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47054 (local address 127.15.11.190:38881)
0809 15:14:19.396944 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:19.396951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:19.397637 (+ 686us) server_negotiation.cc:407] Connection header received
0809 15:14:19.397827 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:19.397833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:19.397927 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:19.398042 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:19.399780 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.400917 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.402736 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.403414 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.403541 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.410784 (+ 7243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47054: BlockingRecv error: recv got EOF from 127.0.0.1:47054 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 15:14:19 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM
I20260809 15:14:19.441548 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.422019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.15.11.190:38881)
0809 15:14:19.422210 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:19.422216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:19.422772 (+ 556us) server_negotiation.cc:407] Connection header received
0809 15:14:19.422967 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:19.422973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:19.423081 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:19.423190 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:19.424887 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.426050 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.428163 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.428816 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.428949 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.434652 (+ 5703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:19.434661 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:19.434666 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:19.434710 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:19.438442 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:19.439294 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:19.439300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:19.439304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:19.439509 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:19.439936 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:19.439943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:19.439946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:19.440329 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:19.440619 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:19.440907 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 15:14:19.441179 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:14:19.445631 23642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:14:19.446427 23642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:14:19.454927 23942 tablet_service.cc:1467] Processing CreateTablet for tablet f4745e3591bd4919b939521213c5d8dd (DEFAULT_TABLE table=test-table [id=2f807c680b874bb1baabed5cdc1625ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:19.455430 23942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4745e3591bd4919b939521213c5d8dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:19.455488 23793 tablet_service.cc:1467] Processing CreateTablet for tablet f4745e3591bd4919b939521213c5d8dd (DEFAULT_TABLE table=test-table [id=2f807c680b874bb1baabed5cdc1625ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:19.455932 23793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4745e3591bd4919b939521213c5d8dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:19.456547 24104 tablet_service.cc:1467] Processing CreateTablet for tablet f4745e3591bd4919b939521213c5d8dd (DEFAULT_TABLE table=test-table [id=2f807c680b874bb1baabed5cdc1625ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:19.456974 24104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4745e3591bd4919b939521213c5d8dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:19.462776 24207 tablet_bootstrap.cc:492] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: Bootstrap starting.
I20260809 15:14:19.464560 24207 tablet_bootstrap.cc:654] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:19.472267 24206 tablet_bootstrap.cc:492] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: Bootstrap starting.
I20260809 15:14:19.473433 24207 tablet_bootstrap.cc:492] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: No bootstrap required, opened a new log
I20260809 15:14:19.473573 24207 ts_tablet_manager.cc:1397] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260809 15:14:19.474129 24206 tablet_bootstrap.cc:654] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:19.474321 24207 raft_consensus.cc:348] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.474606 24207 raft_consensus.cc:374] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:19.474694 24207 raft_consensus.cc:729] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ae7eb3a8994ddb8ae88c9c32e428f9, State: Initialized, Role: FOLLOWER
I20260809 15:14:19.474943 24207 consensus_queue.cc:260] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.478242 24207 ts_tablet_manager.cc:1428] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260809 15:14:19.478374 24208 tablet_bootstrap.cc:492] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: Bootstrap starting.
I20260809 15:14:19.480316 24208 tablet_bootstrap.cc:654] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:19.482689 24208 tablet_bootstrap.cc:492] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: No bootstrap required, opened a new log
I20260809 15:14:19.482884 24208 ts_tablet_manager.cc:1397] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260809 15:14:19.483705 24208 raft_consensus.cc:348] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.484041 24208 raft_consensus.cc:374] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:19.484210 24208 raft_consensus.cc:729] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e56413686b4d6a9b2c04dab129457d, State: Initialized, Role: FOLLOWER
I20260809 15:14:19.484539 24208 consensus_queue.cc:260] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.487285 24208 ts_tablet_manager.cc:1428] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260809 15:14:19.488950 24206 tablet_bootstrap.cc:492] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: No bootstrap required, opened a new log
I20260809 15:14:19.489131 24206 ts_tablet_manager.cc:1397] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.009s
I20260809 15:14:19.490003 24206 raft_consensus.cc:348] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.490373 24206 raft_consensus.cc:374] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:19.490550 24206 raft_consensus.cc:729] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc87148b7e914f9fb35c0e77dbc61fc7, State: Initialized, Role: FOLLOWER
I20260809 15:14:19.490904 24206 consensus_queue.cc:260] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.491571 24206 ts_tablet_manager.cc:1428] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 15:14:19.633962 24213 raft_consensus.cc:482] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:19.634231 24213 raft_consensus.cc:504] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.634945 24213 leader_election.cc:288] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443), bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091)
I20260809 15:14:19.635632 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4745e3591bd4919b939521213c5d8dd" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" is_pre_election: true
I20260809 15:14:19.635607 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4745e3591bd4919b939521213c5d8dd" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" is_pre_election: true
I20260809 15:14:19.635881 24127 raft_consensus.cc:2454] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 0.
I20260809 15:14:19.635893 23960 raft_consensus.cc:2454] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 0.
I20260809 15:14:19.636438 23740 leader_election.cc:302] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ae7eb3a8994ddb8ae88c9c32e428f9, f0e56413686b4d6a9b2c04dab129457d; no voters:
I20260809 15:14:19.636762 24213 raft_consensus.cc:2781] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:14:19.636901 24213 raft_consensus.cc:482] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:14:19.636979 24213 raft_consensus.cc:3037] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:19.639822 24213 raft_consensus.cc:504] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.640427 24213 leader_election.cc:288] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 election: Requested vote from peers 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443), bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091)
I20260809 15:14:19.640906 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4745e3591bd4919b939521213c5d8dd" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9"
I20260809 15:14:19.641119 23960 raft_consensus.cc:3037] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:19.641045 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4745e3591bd4919b939521213c5d8dd" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7"
I20260809 15:14:19.641266 24127 raft_consensus.cc:3037] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:19.643443 23960 raft_consensus.cc:2454] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 1.
I20260809 15:14:19.643894 23740 leader_election.cc:302] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ae7eb3a8994ddb8ae88c9c32e428f9, f0e56413686b4d6a9b2c04dab129457d; no voters:
I20260809 15:14:19.644209 24213 raft_consensus.cc:2781] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:19.644383 24213 raft_consensus.cc:686] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 1 LEADER]: Becoming Leader. State: Replica: f0e56413686b4d6a9b2c04dab129457d, State: Running, Role: LEADER
I20260809 15:14:19.644678 24213 consensus_queue.cc:237] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.644775 24127 raft_consensus.cc:2454] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 1.
I20260809 15:14:19.647827 23642 catalog_manager.cc:5495] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d reported cstate change: term changed from 0 to 1, leader changed from <none> to f0e56413686b4d6a9b2c04dab129457d (127.15.11.129). New cstate: current_term: 1 leader_uuid: "f0e56413686b4d6a9b2c04dab129457d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } health_report { overall_health: UNKNOWN } } }
I20260809 15:14:19.687906 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:19.682599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.15.11.129:32971)
0809 15:14:19.682777 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:19.682785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:19.683013 (+ 228us) server_negotiation.cc:407] Connection header received
0809 15:14:19.683275 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:19.683279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:19.683349 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:19.683460 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:14:19.683850 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.684666 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.685945 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:19.686433 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:19.686577 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:19.687065 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:14:19.687216 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:14:19.687352 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:19.687478 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 15:14:19.687554 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:14:19.690464 24127 raft_consensus.cc:1261] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Refusing update from remote peer f0e56413686b4d6a9b2c04dab129457d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:19.690614 23960 raft_consensus.cc:1261] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Refusing update from remote peer f0e56413686b4d6a9b2c04dab129457d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:19.691102 24239 consensus_queue.cc:1035] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:19.691666 24213 consensus_queue.cc:1035] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:19.706935 23643 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:19.707182 23643 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:19.711184 23643 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf: Sending DeleteTablet for 3 replicas of tablet f4745e3591bd4919b939521213c5d8dd
I20260809 15:14:19.711738 23793 tablet_service.cc:1514] Processing DeleteTablet for tablet f4745e3591bd4919b939521213c5d8dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:19 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:57098
I20260809 15:14:19.712028 24281 tablet_replica.cc:330] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: stopping tablet replica
I20260809 15:14:19.712100 23942 tablet_service.cc:1514] Processing DeleteTablet for tablet f4745e3591bd4919b939521213c5d8dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:19 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:40234
I20260809 15:14:19.712255 24281 raft_consensus.cc:2229] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:14:19.712399 23793 tablet_service.cc:1514] Processing DeleteTablet for tablet f4745e3591bd4919b939521213c5d8dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:19 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:57098
I20260809 15:14:19.712574 24281 raft_consensus.cc:2258] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:19.712898 24282 tablet_replica.cc:330] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: stopping tablet replica
I20260809 15:14:19.712984 23942 tablet_service.cc:1514] Processing DeleteTablet for tablet f4745e3591bd4919b939521213c5d8dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:19 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:40234
I20260809 15:14:19.713109 24282 raft_consensus.cc:2229] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:19.713176 24104 tablet_service.cc:1514] Processing DeleteTablet for tablet f4745e3591bd4919b939521213c5d8dd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:19 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:60388
I20260809 15:14:19.713310 24282 raft_consensus.cc:2258] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:19.713589 24283 tablet_replica.cc:330] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: stopping tablet replica
I20260809 15:14:19.713781 24283 raft_consensus.cc:2229] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:19.714017 24283 raft_consensus.cc:2258] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:19.714390 24281 ts_tablet_manager.cc:1905] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:19.714303 23643 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47068:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:14:19.714947 23643 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:14:19.715705 24283 ts_tablet_manager.cc:1905] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:19.717008 24282 ts_tablet_manager.cc:1905] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:19.722399 24283 ts_tablet_manager.cc:1918] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:19.722553 24283 log.cc:1196] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/f4745e3591bd4919b939521213c5d8dd
I20260809 15:14:19.722977 24283 ts_tablet_manager.cc:1939] T f4745e3591bd4919b939521213c5d8dd P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting consensus metadata
I20260809 15:14:19.723486 24282 ts_tablet_manager.cc:1918] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:19.723491 24281 ts_tablet_manager.cc:1918] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:19.723634 24282 log.cc:1196] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/f4745e3591bd4919b939521213c5d8dd
I20260809 15:14:19.723640 24281 log.cc:1196] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/f4745e3591bd4919b939521213c5d8dd
I20260809 15:14:19.724066 24282 ts_tablet_manager.cc:1939] T f4745e3591bd4919b939521213c5d8dd P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting consensus metadata
I20260809 15:14:19.724076 24281 ts_tablet_manager.cc:1939] T f4745e3591bd4919b939521213c5d8dd P f0e56413686b4d6a9b2c04dab129457d: Deleting consensus metadata
I20260809 15:14:19.724155 23629 catalog_manager.cc:4841] TS bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091): tablet f4745e3591bd4919b939521213c5d8dd (table test-table [id=2f807c680b874bb1baabed5cdc1625ca]) successfully deleted
I20260809 15:14:19.725229 23629 catalog_manager.cc:4841] TS f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971): tablet f4745e3591bd4919b939521213c5d8dd (table test-table [id=2f807c680b874bb1baabed5cdc1625ca]) successfully deleted
I20260809 15:14:19.725561 23627 catalog_manager.cc:4841] TS 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443): tablet f4745e3591bd4919b939521213c5d8dd (table test-table [id=2f807c680b874bb1baabed5cdc1625ca]) successfully deleted
W20260809 15:14:19.727025 23627 catalog_manager.cc:4808] TS 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443): delete failed for tablet f4745e3591bd4919b939521213c5d8dd because the tablet was not found. No further retry: Not found: Tablet not found: f4745e3591bd4919b939521213c5d8dd
W20260809 15:14:19.727154 23629 catalog_manager.cc:4808] TS f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971): delete failed for tablet f4745e3591bd4919b939521213c5d8dd because the tablet was not found. No further retry: Not found: Tablet not found: f4745e3591bd4919b939521213c5d8dd
I20260809 15:14:19.730325 23793 tablet_service.cc:1467] Processing CreateTablet for tablet 63d5a7cff1cb4146a25aa3f7740598da (DEFAULT_TABLE table=test-table [id=b1096967c18e4dab9eb1616231002fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:19.730680 23942 tablet_service.cc:1467] Processing CreateTablet for tablet 63d5a7cff1cb4146a25aa3f7740598da (DEFAULT_TABLE table=test-table [id=b1096967c18e4dab9eb1616231002fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:19.730790 23793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d5a7cff1cb4146a25aa3f7740598da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:19.730911 24104 tablet_service.cc:1467] Processing CreateTablet for tablet 63d5a7cff1cb4146a25aa3f7740598da (DEFAULT_TABLE table=test-table [id=b1096967c18e4dab9eb1616231002fea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:19.731040 23942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d5a7cff1cb4146a25aa3f7740598da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:19.731294 24104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d5a7cff1cb4146a25aa3f7740598da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:19.737289 24207 tablet_bootstrap.cc:492] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: Bootstrap starting.
I20260809 15:14:19.737407 24206 tablet_bootstrap.cc:492] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: Bootstrap starting.
I20260809 15:14:19.738729 24208 tablet_bootstrap.cc:492] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: Bootstrap starting.
I20260809 15:14:19.738868 24207 tablet_bootstrap.cc:654] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:19.738868 24206 tablet_bootstrap.cc:654] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:19.740458 24208 tablet_bootstrap.cc:654] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:19.751211 24207 tablet_bootstrap.cc:492] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: No bootstrap required, opened a new log
I20260809 15:14:19.751461 24207 ts_tablet_manager.cc:1397] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20260809 15:14:19.751823 24206 tablet_bootstrap.cc:492] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: No bootstrap required, opened a new log
I20260809 15:14:19.752034 24206 ts_tablet_manager.cc:1397] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.013s
I20260809 15:14:19.752460 24207 raft_consensus.cc:348] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.753073 24207 raft_consensus.cc:374] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:19.752969 24206 raft_consensus.cc:348] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.753226 24206 raft_consensus.cc:374] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:19.753317 24206 raft_consensus.cc:729] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc87148b7e914f9fb35c0e77dbc61fc7, State: Initialized, Role: FOLLOWER
I20260809 15:14:19.753268 24207 raft_consensus.cc:729] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ae7eb3a8994ddb8ae88c9c32e428f9, State: Initialized, Role: FOLLOWER
I20260809 15:14:19.753616 24206 consensus_queue.cc:260] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.753676 24207 consensus_queue.cc:260] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.755172 24206 ts_tablet_manager.cc:1428] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 15:14:19.756063 24208 tablet_bootstrap.cc:492] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: No bootstrap required, opened a new log
I20260809 15:14:19.756266 24208 ts_tablet_manager.cc:1397] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20260809 15:14:19.757036 24207 ts_tablet_manager.cc:1428] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 15:14:19.757051 24208 raft_consensus.cc:348] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.757328 24208 raft_consensus.cc:374] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:19.757413 24208 raft_consensus.cc:729] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e56413686b4d6a9b2c04dab129457d, State: Initialized, Role: FOLLOWER
I20260809 15:14:19.758888 24208 consensus_queue.cc:260] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.759493 24208 ts_tablet_manager.cc:1428] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:14:19.942070 24213 raft_consensus.cc:482] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:19.942267 24213 raft_consensus.cc:504] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.942975 24213 leader_election.cc:288] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443), bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091)
I20260809 15:14:19.943445 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d5a7cff1cb4146a25aa3f7740598da" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" is_pre_election: true
I20260809 15:14:19.943504 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d5a7cff1cb4146a25aa3f7740598da" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" is_pre_election: true
I20260809 15:14:19.943691 23960 raft_consensus.cc:2454] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 0.
I20260809 15:14:19.943737 24127 raft_consensus.cc:2454] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 0.
I20260809 15:14:19.944106 23740 leader_election.cc:302] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ae7eb3a8994ddb8ae88c9c32e428f9, f0e56413686b4d6a9b2c04dab129457d; no voters:
I20260809 15:14:19.944420 24213 raft_consensus.cc:2781] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:14:19.944546 24213 raft_consensus.cc:482] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:14:19.944669 24213 raft_consensus.cc:3037] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:19.946777 24213 raft_consensus.cc:504] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.947340 24213 leader_election.cc:288] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 election: Requested vote from peers 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443), bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091)
I20260809 15:14:19.947762 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d5a7cff1cb4146a25aa3f7740598da" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9"
I20260809 15:14:19.947808 24127 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d5a7cff1cb4146a25aa3f7740598da" candidate_uuid: "f0e56413686b4d6a9b2c04dab129457d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7"
I20260809 15:14:19.947949 23960 raft_consensus.cc:3037] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:19.948038 24127 raft_consensus.cc:3037] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:19.949954 23960 raft_consensus.cc:2454] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 1.
I20260809 15:14:19.949965 24127 raft_consensus.cc:2454] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0e56413686b4d6a9b2c04dab129457d in term 1.
I20260809 15:14:19.950338 23740 leader_election.cc:302] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ae7eb3a8994ddb8ae88c9c32e428f9, f0e56413686b4d6a9b2c04dab129457d; no voters:
I20260809 15:14:19.950627 24213 raft_consensus.cc:2781] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:19.950841 24213 raft_consensus.cc:686] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 1 LEADER]: Becoming Leader. State: Replica: f0e56413686b4d6a9b2c04dab129457d, State: Running, Role: LEADER
I20260809 15:14:19.951102 24213 consensus_queue.cc:237] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } }
I20260809 15:14:19.953612 23642 catalog_manager.cc:5495] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d reported cstate change: term changed from 0 to 1, leader changed from <none> to f0e56413686b4d6a9b2c04dab129457d (127.15.11.129). New cstate: current_term: 1 leader_uuid: "f0e56413686b4d6a9b2c04dab129457d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0e56413686b4d6a9b2c04dab129457d" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 32971 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 } health_report { overall_health: UNKNOWN } } }
I20260809 15:14:20.001642 24127 raft_consensus.cc:1261] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Refusing update from remote peer f0e56413686b4d6a9b2c04dab129457d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:20.001740 23960 raft_consensus.cc:1261] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Refusing update from remote peer f0e56413686b4d6a9b2c04dab129457d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:20.002234 24213 consensus_queue.cc:1035] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc87148b7e914f9fb35c0e77dbc61fc7" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 37091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:20.002528 24239 consensus_queue.cc:1035] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ae7eb3a8994ddb8ae88c9c32e428f9" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:20.043524 23642 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:20.043818 23642 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47068:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:20.046548 23642 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fab7693283d433783e9bc39ae1b50bf: Sending DeleteTablet for 3 replicas of tablet 63d5a7cff1cb4146a25aa3f7740598da
I20260809 15:14:20.047268 23793 tablet_service.cc:1514] Processing DeleteTablet for tablet 63d5a7cff1cb4146a25aa3f7740598da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:20 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:57098
I20260809 15:14:20.047472 23942 tablet_service.cc:1514] Processing DeleteTablet for tablet 63d5a7cff1cb4146a25aa3f7740598da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:20 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:40234
I20260809 15:14:20.047523 24104 tablet_service.cc:1514] Processing DeleteTablet for tablet 63d5a7cff1cb4146a25aa3f7740598da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:20 UTC) from {username='oryx', principal='oryx/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:60388
I20260809 15:14:20.047586 24281 tablet_replica.cc:330] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: stopping tablet replica
I20260809 15:14:20.047746 24282 tablet_replica.cc:330] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: stopping tablet replica
I20260809 15:14:20.047770 24281 raft_consensus.cc:2229] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:14:20.047923 24282 raft_consensus.cc:2229] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:20.048058 24283 tablet_replica.cc:330] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: stopping tablet replica
I20260809 15:14:20.048111 24282 raft_consensus.cc:2258] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:20.048099 24281 raft_consensus.cc:2258] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:20.048266 24283 raft_consensus.cc:2229] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:20.048482 24283 raft_consensus.cc:2258] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:20.048816 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 23724
I20260809 15:14:20.049350 24281 ts_tablet_manager.cc:1905] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:20.049852 24282 ts_tablet_manager.cc:1905] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:20.050772 24283 ts_tablet_manager.cc:1905] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:20.057384 24282 ts_tablet_manager.cc:1918] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:14:20.057384 24283 ts_tablet_manager.cc:1918] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:14:20.057513 24283 log.cc:1196] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/63d5a7cff1cb4146a25aa3f7740598da
I20260809 15:14:20.057523 24282 log.cc:1196] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/63d5a7cff1cb4146a25aa3f7740598da
I20260809 15:14:20.057966 24282 ts_tablet_manager.cc:1939] T 63d5a7cff1cb4146a25aa3f7740598da P 44ae7eb3a8994ddb8ae88c9c32e428f9: Deleting consensus metadata
I20260809 15:14:20.058002 24283 ts_tablet_manager.cc:1939] T 63d5a7cff1cb4146a25aa3f7740598da P bc87148b7e914f9fb35c0e77dbc61fc7: Deleting consensus metadata
I20260809 15:14:20.058082 24281 ts_tablet_manager.cc:1918] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:14:20.058234 24281 log.cc:1196] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/63d5a7cff1cb4146a25aa3f7740598da
I20260809 15:14:20.058751 24281 ts_tablet_manager.cc:1939] T 63d5a7cff1cb4146a25aa3f7740598da P f0e56413686b4d6a9b2c04dab129457d: Deleting consensus metadata
I20260809 15:14:20.059146 23627 catalog_manager.cc:4841] TS 44ae7eb3a8994ddb8ae88c9c32e428f9 (127.15.11.130:43443): tablet 63d5a7cff1cb4146a25aa3f7740598da (table test-table [id=b1096967c18e4dab9eb1616231002fea]) successfully deleted
I20260809 15:14:20.059355 23629 catalog_manager.cc:4841] TS bc87148b7e914f9fb35c0e77dbc61fc7 (127.15.11.131:37091): tablet 63d5a7cff1cb4146a25aa3f7740598da (table test-table [id=b1096967c18e4dab9eb1616231002fea]) successfully deleted
I20260809 15:14:20.059950 23629 catalog_manager.cc:4841] TS f0e56413686b4d6a9b2c04dab129457d (127.15.11.129:32971): tablet 63d5a7cff1cb4146a25aa3f7740598da (table test-table [id=b1096967c18e4dab9eb1616231002fea]) successfully deleted
Aug 09 15:14:20 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.129@KRBTEST.COM
I20260809 15:14:20.073861 24269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:20.055818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57132 (local address 127.15.11.129:32971)
0809 15:14:20.056019 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:20.056026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:20.059848 (+ 3822us) server_negotiation.cc:407] Connection header received
0809 15:14:20.060118 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:20.060162 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:20.060239 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:20.060346 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:20.061608 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:20.062363 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:20.063704 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:20.064174 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:20.064309 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:20.068674 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:20.068691 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:20.068720 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:20.068754 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:20.071312 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:20.071951 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:20.071958 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:20.071960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:20.072078 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:20.072476 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:20.072480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:20.072481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:20.072833 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:20.073061 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:20.073317 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 15:14:20.073511 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:14:20.074548 23863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:20.201581 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 23724
I20260809 15:14:20.230809 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 23872
Aug 09 15:14:20 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.130@KRBTEST.COM
I20260809 15:14:20.246646 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:20.231703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.15.11.130:43443)
0809 15:14:20.232273 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:20.232277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:20.232293 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:14:20.232347 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:20.232351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:20.232427 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:20.232517 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:20.234095 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:20.234845 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:20.236799 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:20.237211 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:20.237303 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:20.241497 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:20.241511 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:20.241526 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:20.241562 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:20.244361 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:20.244945 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:20.244952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:20.244954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:20.245107 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:20.245417 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:20.245421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:20.245422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:20.245730 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:20.246110 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:20.246229 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 15:14:20.246392 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":279,"threads_started":1}
I20260809 15:14:20.247335 24015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:20.344107 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 23872
I20260809 15:14:20.366986 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 24026
Aug 09 15:14:20 dist-test-slave-8w9v krb5kdc[23594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.11.131@KRBTEST.COM
I20260809 15:14:20.382557 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:20.367868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.15.11.131:37091)
0809 15:14:20.368360 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:20.368366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:20.368385 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:14:20.368440 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:20.368444 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:20.368517 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:20.368610 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:20.370123 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:20.370797 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:20.372097 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:20.372556 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:20.372650 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:20.377336 (+ 4686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:20.377353 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:20.377371 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:20.377407 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:20.380356 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:20.380978 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:20.380985 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:20.380987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:20.381105 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:20.381400 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:20.381403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:20.381405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:20.381715 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:20.381919 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:20.382148 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 15:14:20.382296 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":310,"threads_started":1}
I20260809 15:14:20.383220 24172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:20.527400 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 24026
I20260809 15:14:20.558666 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 23608
I20260809 15:14:20.559840 23689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:20.677162 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 23608
2026-08-09T15:14:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6975 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:14:20 dist-test-slave-8w9v krb5kdc[24331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:14:20 dist-test-slave-8w9v krb5kdc[24331](info): set up 2 sockets
Aug 09 15:14:20 dist-test-slave-8w9v krb5kdc[24331](info): commencing operation
krb5kdc: starting...
W20260809 15:14:21.824697 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.007s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:14:21 dist-test-slave-8w9v krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:14:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:14:22.083460 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:40173
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:40173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:22.417781 24348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:22.418078 24348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:22.418192 24348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:22.424962 24348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:22.425066 24348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:22.425109 24348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:22.425163 24348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:22.425206 24348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:22.434501 24348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:40173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:40173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:22.435775 24348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:22.437137 24348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:22.449183 24353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:22.449211 24356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:22.449121 24354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:22.449615 24348 server_base.cc:1034] running on GCE node
I20260809 15:14:22.450502 24348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:22.451831 24348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:22.453099 24348 hybrid_clock.cc:648] HybridClock initialized: now 1786288462453017 us; error 92 us; skew 500 ppm
I20260809 15:14:22.453447 24348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T15:14:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1848 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:14:22 dist-test-slave-8w9v krb5kdc[24362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:14:22 dist-test-slave-8w9v krb5kdc[24362](info): set up 2 sockets
Aug 09 15:14:22 dist-test-slave-8w9v krb5kdc[24362](info): commencing operation
krb5kdc: starting...
W20260809 15:14:23.681998 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:14:23 dist-test-slave-8w9v krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:14:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:14:23.920519 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35905
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:35905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:24.254979 24378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:24.255332 24378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:24.255515 24378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:24.262545 24378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:24.262710 24378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:24.262799 24378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:24.262885 24378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:24.262952 24378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:24.272745 24378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:35905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:24.274077 24378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:24.275910 24378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:24.287844 24384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:24.287909 24388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:24.287914 24383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:24.288816 24378 server_base.cc:1034] running on GCE node
I20260809 15:14:24.289521 24378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:24.290823 24378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:24.292032 24378 hybrid_clock.cc:648] HybridClock initialized: now 1786288464291990 us; error 28 us; skew 500 ppm
I20260809 15:14:24.292425 24378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:24 dist-test-slave-8w9v krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288464, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:24.295565 24378 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:14:24.297134 24378 webserver.cc:460] Webserver started at http://127.15.11.190:38775/ using document root <none> and password file <none>
I20260809 15:14:24.297829 24378 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:24.297959 24378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:24.298316 24378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:24.301404 24378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "034df6000e114e9387fb2023002b04ba"
format_stamp: "Formatted at 2026-08-09 15:14:24 on dist-test-slave-8w9v"
server_key: "86c2f8abb5acd2a66e3a5802bc5172d9"
server_key_iv: "631e43682e41a5c9be352dc70cb12b02"
server_key_version: "encryptionkey@0"
I20260809 15:14:24.302188 24378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "034df6000e114e9387fb2023002b04ba"
format_stamp: "Formatted at 2026-08-09 15:14:24 on dist-test-slave-8w9v"
server_key: "86c2f8abb5acd2a66e3a5802bc5172d9"
server_key_iv: "631e43682e41a5c9be352dc70cb12b02"
server_key_version: "encryptionkey@0"
I20260809 15:14:24.307825 24378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:14:24.311169 24396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:24.312485 24378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:24.312685 24378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "034df6000e114e9387fb2023002b04ba"
format_stamp: "Formatted at 2026-08-09 15:14:24 on dist-test-slave-8w9v"
server_key: "86c2f8abb5acd2a66e3a5802bc5172d9"
server_key_iv: "631e43682e41a5c9be352dc70cb12b02"
server_key_version: "encryptionkey@0"
I20260809 15:14:24.312868 24378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:24.325542 24378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:24.329386 24378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:24.329671 24378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:24.347873 24378 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:35905
I20260809 15:14:24.347894 24454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:35905 every 8 connection(s)
I20260809 15:14:24.349261 24378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:14:24.352936 24456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:24.353250 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 24378
I20260809 15:14:24.353487 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:14:24.354182 15406 external_mini_cluster.cc:1428] Setting key ace8d2819f86f88c44107228967b58f3
I20260809 15:14:24.367388 24456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: Bootstrap starting.
Aug 09 15:14:24 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:24.370573 24456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:24.371979 24456 log.cc:824] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:24.375526 24456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: No bootstrap required, opened a new log
I20260809 15:14:24.379341 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:24.356267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.15.11.190:35905)
0809 15:14:24.357110 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:24.357125 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:24.357182 (+ 57us) server_negotiation.cc:407] Connection header received
0809 15:14:24.359725 (+ 2543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:24.359752 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:24.360258 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:24.360712 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:24.362030 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:24.364008 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:24.365302 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:24.366362 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:24.366559 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:24.370444 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:24.370462 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:24.370485 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:24.370526 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:24.374554 (+ 4028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:24.375311 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:24.375319 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:24.375326 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:24.375542 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:24.376037 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:24.376042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:24.376045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:24.376777 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:24.377071 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:24.377561 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 15:14:24.377997 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":328,"threads_started":1}
I20260809 15:14:24.384958 24456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } }
I20260809 15:14:24.385278 24456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:24.385402 24456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034df6000e114e9387fb2023002b04ba, State: Initialized, Role: FOLLOWER
I20260809 15:14:24.385946 24456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } }
I20260809 15:14:24.386142 24456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:14:24.386256 24456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:14:24.386374 24456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:24.388466 24456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } }
I20260809 15:14:24.388863 24456 leader_election.cc:302] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 034df6000e114e9387fb2023002b04ba; no voters:
I20260809 15:14:24.389456 24456 leader_election.cc:288] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:14:24.389637 24461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:24.390331 24461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 1 LEADER]: Becoming Leader. State: Replica: 034df6000e114e9387fb2023002b04ba, State: Running, Role: LEADER
I20260809 15:14:24.390893 24461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } }
I20260809 15:14:24.391467 24456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:14:24.393838 24462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "034df6000e114e9387fb2023002b04ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } } }
I20260809 15:14:24.394613 24462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:24.393815 24463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 034df6000e114e9387fb2023002b04ba. Latest consensus state: current_term: 1 leader_uuid: "034df6000e114e9387fb2023002b04ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } } }
I20260809 15:14:24.394936 24463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:24.397074 24467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:14:24.403939 24467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:14:24.413486 24467 catalog_manager.cc:1347] Generated new cluster ID: 8ec5199d6036411890cd40cccc06f975
I20260809 15:14:24.413597 24467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:14:24.438534 24467 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:14:24.439918 24467 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:14:24.457340 24467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: Generated new TSK 0
I20260809 15:14:24.458218 24467 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:14:24.523705 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35905
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:24.866052 24483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:24.866390 24483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:24.866569 24483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:24.873929 24483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:24.874105 24483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:24.874280 24483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:14:24.884696 24483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:24.886003 24483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:24.887442 24483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:24.900849 24494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:24.900806 24491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:24.900904 24490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:24.901716 24483 server_base.cc:1034] running on GCE node
I20260809 15:14:24.902443 24483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:24.903407 24483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:24.904604 24483 hybrid_clock.cc:648] HybridClock initialized: now 1786288464904565 us; error 29 us; skew 500 ppm
I20260809 15:14:24.904873 24483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:24 dist-test-slave-8w9v krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288464, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:24.908008 24483 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:14:24.909590 24483 webserver.cc:460] Webserver started at http://127.15.11.129:34233/ using document root <none> and password file <none>
I20260809 15:14:24.910274 24483 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:24.910406 24483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:24.910728 24483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:24.913661 24483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "92d063391b224f1b8765df08025f1489"
format_stamp: "Formatted at 2026-08-09 15:14:24 on dist-test-slave-8w9v"
server_key: "d8c59e5cf6027392d56273a184d7fe7b"
server_key_iv: "9a30860b7751edf968aec91733d7dd03"
server_key_version: "encryptionkey@0"
I20260809 15:14:24.914739 24483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "92d063391b224f1b8765df08025f1489"
format_stamp: "Formatted at 2026-08-09 15:14:24 on dist-test-slave-8w9v"
server_key: "d8c59e5cf6027392d56273a184d7fe7b"
server_key_iv: "9a30860b7751edf968aec91733d7dd03"
server_key_version: "encryptionkey@0"
I20260809 15:14:24.920789 24483 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 15:14:24.924381 24501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:24.925698 24483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:14:24.925895 24483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "92d063391b224f1b8765df08025f1489"
format_stamp: "Formatted at 2026-08-09 15:14:24 on dist-test-slave-8w9v"
server_key: "d8c59e5cf6027392d56273a184d7fe7b"
server_key_iv: "9a30860b7751edf968aec91733d7dd03"
server_key_version: "encryptionkey@0"
I20260809 15:14:24.926123 24483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:24.957557 24483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:24.961758 24483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:24.962059 24483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:24.963018 24483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:24.964386 24483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:24.964496 24483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:24.964633 24483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:24.964709 24483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:24.998579 24483 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:38011
I20260809 15:14:24.998626 24622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:38011 every 8 connection(s)
I20260809 15:14:24.999914 24483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:14:25.000676 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 24483
I20260809 15:14:25.000905 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:14:25.001742 15406 external_mini_cluster.cc:1428] Setting key f2efb476dc2859b8ff48598baefdd451
Aug 09 15:14:25 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288464, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:25.021812 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:25.002979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:58073 (local address 127.15.11.190:35905)
0809 15:14:25.003417 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:25.003422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:25.004730 (+ 1308us) server_negotiation.cc:407] Connection header received
0809 15:14:25.006651 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:25.006660 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:25.006752 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:25.006871 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:25.008429 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:25.009574 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:25.011123 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:25.011843 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:25.011988 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:25.015796 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:25.015801 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:25.015805 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:25.015835 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:25.018721 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:25.019449 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:25.019453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:25.019455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:25.019570 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:25.020001 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:25.020006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:25.020011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:25.020336 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:25.020499 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:25.021184 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 15:14:25.021434 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":239,"threads_started":1}
I20260809 15:14:25.022794 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:25.003614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:58073)
0809 15:14:25.004547 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:25.004603 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:25.006377 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:25.007086 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:25.007093 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:25.007606 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:25.008251 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:25.008272 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:25.009733 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:25.009736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:25.010976 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:25.010988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:25.012051 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:25.012056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:25.012328 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:25.013034 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:25.013056 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:25.015586 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:25.018867 (+ 3281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:25.018876 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:25.018898 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:25.019310 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:25.019681 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:25.019686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:25.019689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:25.019877 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:25.020473 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:25.020480 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:25.020811 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 15:14:25.021075 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 15:14:25.021604 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":442,"threads_started":1}
I20260809 15:14:25.024547 24624 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35905
I20260809 15:14:25.024847 24624 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:25.025612 24624 heartbeater.cc:511] Master 127.15.11.190:35905 requested a full tablet report, sending...
I20260809 15:14:25.028214 24415 ts_manager.cc:194] Registered new tserver with Master: 92d063391b224f1b8765df08025f1489 (127.15.11.129:38011)
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
I20260809 15:14:25.029961 24415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:58073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:14:25.061252 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35905
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:25.393668 24630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:25.393950 24630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:25.394145 24630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:25.401345 24630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:25.401481 24630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:25.401657 24630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:14:25.411008 24630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:25.412278 24630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:25.413548 24630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:25.426416 24637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:25.426507 24636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:25.426508 24641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:25.427461 24630 server_base.cc:1034] running on GCE node
I20260809 15:14:25.428076 24630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:25.428967 24630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:25.430158 24630 hybrid_clock.cc:648] HybridClock initialized: now 1786288465430118 us; error 51 us; skew 500 ppm
I20260809 15:14:25.430423 24630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:25 dist-test-slave-8w9v krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288465, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:25.433537 24630 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:14:25.434916 24630 webserver.cc:460] Webserver started at http://127.15.11.130:33871/ using document root <none> and password file <none>
I20260809 15:14:25.435585 24630 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:25.435729 24630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:25.436071 24630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:25.438928 24630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "9fca5d1db3ae43e9afbfd7af184c6e2c"
format_stamp: "Formatted at 2026-08-09 15:14:25 on dist-test-slave-8w9v"
server_key: "4bebf98ff436fc023d398e23e1d1ccee"
server_key_iv: "3159b00e85933c309051931bcb53494d"
server_key_version: "encryptionkey@0"
I20260809 15:14:25.439798 24630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "9fca5d1db3ae43e9afbfd7af184c6e2c"
format_stamp: "Formatted at 2026-08-09 15:14:25 on dist-test-slave-8w9v"
server_key: "4bebf98ff436fc023d398e23e1d1ccee"
server_key_iv: "3159b00e85933c309051931bcb53494d"
server_key_version: "encryptionkey@0"
I20260809 15:14:25.444883 24630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:14:25.448175 24648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:25.449368 24630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:14:25.449543 24630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "9fca5d1db3ae43e9afbfd7af184c6e2c"
format_stamp: "Formatted at 2026-08-09 15:14:25 on dist-test-slave-8w9v"
server_key: "4bebf98ff436fc023d398e23e1d1ccee"
server_key_iv: "3159b00e85933c309051931bcb53494d"
server_key_version: "encryptionkey@0"
I20260809 15:14:25.449751 24630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:25.465157 24630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:25.469007 24630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:25.469308 24630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:25.470230 24630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:25.471601 24630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:25.471709 24630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:25.471869 24630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:25.471949 24630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:25.506304 24630 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:34833
I20260809 15:14:25.506398 24771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:34833 every 8 connection(s)
I20260809 15:14:25.507653 24630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:14:25.513839 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 24630
I20260809 15:14:25.514035 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:14:25.514611 15406 external_mini_cluster.cc:1428] Setting key 61c1d3a5de1cd6281713a409cbfbe6c4
Aug 09 15:14:25 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288465, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:25.529903 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:25.510411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:40143 (local address 127.15.11.190:35905)
0809 15:14:25.510587 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:25.510592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:25.511688 (+ 1096us) server_negotiation.cc:407] Connection header received
0809 15:14:25.513641 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:25.513646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:25.513719 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:25.513807 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:25.515745 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:25.516676 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:25.518312 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:25.519144 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:25.519298 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:25.522950 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:25.522959 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:25.522967 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:25.523010 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:25.526693 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:25.527411 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:25.527417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:25.527425 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:25.527558 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:25.528027 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:25.528032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:25.528034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:25.528380 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:25.528539 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:25.529264 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 15:14:25.529528 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:14:25.530943 24774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:25.510807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:40143)
0809 15:14:25.511524 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:25.511561 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:25.513311 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:25.514120 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:25.514131 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:25.514865 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:25.515562 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:25.515579 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:25.516853 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:25.516857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:25.518170 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:25.518182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:25.519372 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:25.519377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:25.519574 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:25.520341 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:25.520364 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:25.522721 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:25.526829 (+ 4108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:25.526836 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:25.526850 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:25.527272 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:25.527675 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:25.527681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:25.527686 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:25.527893 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:25.528514 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:25.528522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:25.528884 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 15:14:25.529155 (+ 271us) client_negotiation.cc:240] Negotiation successful
0809 15:14:25.529717 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":273,"threads_started":1}
I20260809 15:14:25.532533 24772 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35905
I20260809 15:14:25.532855 24772 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:25.533726 24772 heartbeater.cc:511] Master 127.15.11.190:35905 requested a full tablet report, sending...
I20260809 15:14:25.535379 24415 ts_manager.cc:194] Registered new tserver with Master: 9fca5d1db3ae43e9afbfd7af184c6e2c (127.15.11.130:34833)
I20260809 15:14:25.536252 24415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:40143
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:14:25.573704 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35905
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:25.922107 24778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:25.922420 24778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:25.922621 24778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:25.929997 24778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:25.930188 24778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:25.930331 24778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:14:25.939977 24778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:25.941224 24778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:25.942538 24778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:25.955906 24784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:25.955876 24788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:25.955962 24785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:25.956620 24778 server_base.cc:1034] running on GCE node
I20260809 15:14:25.957190 24778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:25.958146 24778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:25.959376 24778 hybrid_clock.cc:648] HybridClock initialized: now 1786288465959337 us; error 52 us; skew 500 ppm
I20260809 15:14:25.959681 24778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:25 dist-test-slave-8w9v krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288465, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:25.962895 24778 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:14:25.964359 24778 webserver.cc:460] Webserver started at http://127.15.11.131:39211/ using document root <none> and password file <none>
I20260809 15:14:25.965019 24778 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:25.965171 24778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:25.965482 24778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:25.968323 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "14190964039f40a092aa0a276701e5e9"
format_stamp: "Formatted at 2026-08-09 15:14:25 on dist-test-slave-8w9v"
server_key: "50005c1b01f1d710a271c045820764b3"
server_key_iv: "e8bb63f784c9dfb9922536796b848805"
server_key_version: "encryptionkey@0"
I20260809 15:14:25.969092 24778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "14190964039f40a092aa0a276701e5e9"
format_stamp: "Formatted at 2026-08-09 15:14:25 on dist-test-slave-8w9v"
server_key: "50005c1b01f1d710a271c045820764b3"
server_key_iv: "e8bb63f784c9dfb9922536796b848805"
server_key_version: "encryptionkey@0"
I20260809 15:14:25.973907 24778 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 15:14:25.977267 24796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:25.978519 24778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:14:25.978698 24778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "14190964039f40a092aa0a276701e5e9"
format_stamp: "Formatted at 2026-08-09 15:14:25 on dist-test-slave-8w9v"
server_key: "50005c1b01f1d710a271c045820764b3"
server_key_iv: "e8bb63f784c9dfb9922536796b848805"
server_key_version: "encryptionkey@0"
I20260809 15:14:25.978912 24778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:26.001885 24778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:26.006313 24778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:26.006629 24778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:26.007567 24778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:26.009006 24778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:26.009114 24778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:26.009266 24778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:26.009346 24778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:26.033591 24624 heartbeater.cc:503] Master 127.15.11.190:35905 was elected leader, sending a full tablet report...
I20260809 15:14:26.043730 24778 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:40641
I20260809 15:14:26.043781 24919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:40641 every 8 connection(s)
I20260809 15:14:26.045058 24778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:14:26.049427 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 24778
I20260809 15:14:26.049604 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:14:26.050056 15406 external_mini_cluster.cc:1428] Setting key 7a2a76312bdbfd3a885bea6fa82d4e99
Aug 09 15:14:26 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288465, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:26.065069 24922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:26.047583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:41983 (local address 127.15.11.190:35905)
0809 15:14:26.048049 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:26.048053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:26.048967 (+ 914us) server_negotiation.cc:407] Connection header received
0809 15:14:26.050974 (+ 2007us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:26.050980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:26.051063 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:26.051185 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:26.052893 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.053774 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.055389 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.055799 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.055936 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:26.059406 (+ 3470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:26.059411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:26.059414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:26.059446 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:26.062035 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:26.062705 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:26.062709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:26.062711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:26.062831 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:26.063188 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:26.063192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:26.063193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:26.063443 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:26.063578 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:26.064453 (+ 875us) server_negotiation.cc:299] Negotiation successful
0809 15:14:26.064680 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":270,"threads_started":1}
I20260809 15:14:26.065918 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:26.048056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:41983)
0809 15:14:26.048803 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:26.048847 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:26.050660 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:26.051466 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:26.051478 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:26.052112 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:26.052728 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:26.052745 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.053937 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.053942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:26.055233 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:26.055245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.056046 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.056055 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:26.056334 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:26.057040 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:26.057062 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:26.059201 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:26.062160 (+ 2959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:26.062166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:26.062176 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:26.062598 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:26.062926 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:26.062929 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:26.062931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:26.063093 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:26.063580 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:26.063590 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:26.063963 (+ 373us) client_negotiation.cc:769] Sending connection context
0809 15:14:26.064288 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 15:14:26.064727 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":264,"threads_started":1}
I20260809 15:14:26.067198 24920 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35905
I20260809 15:14:26.067618 24920 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:26.068403 24920 heartbeater.cc:511] Master 127.15.11.190:35905 requested a full tablet report, sending...
I20260809 15:14:26.069821 24415 ts_manager.cc:194] Registered new tserver with Master: 14190964039f40a092aa0a276701e5e9 (127.15.11.131:40641)
I20260809 15:14:26.070662 24415 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:41983
I20260809 15:14:26.077481 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab.
Entry for principal oryx/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab.
WARNING: no policy specified for oryx/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab.
Entry for principal oryx/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab.
WARNING: no policy specified for oryx/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab.
Entry for principal oryx/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab.
I20260809 15:14:26.226505 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 24483
Aug 09 15:14:26 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:14:26.242731 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:26.227852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51428 (local address 127.15.11.129:38011)
0809 15:14:26.228627 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:26.228633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:26.228661 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:14:26.228724 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:26.228731 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:26.228965 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:26.229099 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:26.230505 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.231589 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.232803 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.233391 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.233507 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:26.237492 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:26.237504 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:26.237518 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:26.237556 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:26.240250 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:26.240883 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:26.240891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:26.240893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:26.241037 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:26.241362 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:26.241377 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:26.241379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:26.241752 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:26.242018 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:26.242210 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 15:14:26.242422 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":428,"threads_started":1}
I20260809 15:14:26.244118 24618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:26.326453 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 24483
I20260809 15:14:26.347451 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 24630
Aug 09 15:14:26 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:14:26.364081 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:26.348405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42044 (local address 127.15.11.130:34833)
0809 15:14:26.348973 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:26.348980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:26.349005 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:14:26.349067 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:26.349073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:26.349277 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:26.349478 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:26.350701 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.351658 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.353104 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.353931 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.354051 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:26.358401 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:26.358412 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:26.358424 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:26.358456 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:26.361238 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:26.361926 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:26.361933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:26.361935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:26.362068 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:26.362490 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:26.362494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:26.362496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:26.362837 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:26.363175 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:26.363477 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 15:14:26.363737 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":309,"threads_started":1}
I20260809 15:14:26.365360 24767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:26.447106 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 24630
I20260809 15:14:26.467185 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 24778
Aug 09 15:14:26 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:14:26.483654 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:26.468106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58874 (local address 127.15.11.131:40641)
0809 15:14:26.468738 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:26.468744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:26.468771 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:14:26.468832 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:26.468839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:26.469040 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:26.469220 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:26.470538 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.471563 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.472956 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:26.473511 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:26.473662 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:26.478232 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:26.478244 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:26.478258 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:26.478295 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:26.480932 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:26.481596 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:26.481603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:26.481605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:26.481729 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:26.482107 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:26.482111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:26.482113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:26.482456 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:26.482773 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:26.483070 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 15:14:26.483327 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":284,"threads_started":1}
I20260809 15:14:26.485071 24915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:26.568539 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 24778
I20260809 15:14:26.589776 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 24378
I20260809 15:14:26.590970 24448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:26.696839 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 24378
I20260809 15:14:26.721673 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35905
--webserver_interface=127.15.11.190
--webserver_port=38775
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:35905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:27.053228 24939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:27.053511 24939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:27.053684 24939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:27.060798 24939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:27.060899 24939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:27.061007 24939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:27.061103 24939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:27.061188 24939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:27.070839 24939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:35905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=38775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:27.072245 24939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:27.073580 24939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:27.085454 24945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:27.085413 24948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:27.085553 24946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:27.086215 24939 server_base.cc:1034] running on GCE node
I20260809 15:14:27.086939 24939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:27.088306 24939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:27.089512 24939 hybrid_clock.cc:648] HybridClock initialized: now 1786288467089461 us; error 44 us; skew 500 ppm
I20260809 15:14:27.089879 24939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:27 dist-test-slave-8w9v krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288467, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:27.093254 24939 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:14:27.094779 24939 webserver.cc:460] Webserver started at http://127.15.11.190:38775/ using document root <none> and password file <none>
I20260809 15:14:27.095491 24939 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:27.095654 24939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:27.101116 24939 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:27.104044 24956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:27.105386 24939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:14:27.105573 24939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "034df6000e114e9387fb2023002b04ba"
format_stamp: "Formatted at 2026-08-09 15:14:24 on dist-test-slave-8w9v"
server_key: "86c2f8abb5acd2a66e3a5802bc5172d9"
server_key_iv: "631e43682e41a5c9be352dc70cb12b02"
server_key_version: "encryptionkey@0"
I20260809 15:14:27.106550 24939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:27.134951 24939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:27.139658 24939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:27.139963 24939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:27.159235 24939 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:35905
I20260809 15:14:27.159253 25016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:35905 every 8 connection(s)
I20260809 15:14:27.160823 24939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:14:27.167124 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 24939
I20260809 15:14:27.168188 25017 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:14:27.168216 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:38011
--local_ip_for_outbound_sockets=127.15.11.129
--tserver_master_addrs=127.15.11.190:35905
--webserver_port=34233
--webserver_interface=127.15.11.129
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.11.129
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:14:27.173432 25017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: Bootstrap starting.
I20260809 15:14:27.194581 25017 log.cc:824] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:27.203666 25017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:14:27.204289 25017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: Bootstrap complete.
I20260809 15:14:27.214074 25017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } }
I20260809 15:14:27.214939 25017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 034df6000e114e9387fb2023002b04ba, State: Initialized, Role: FOLLOWER
I20260809 15:14:27.215574 25017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } }
I20260809 15:14:27.215817 25017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:14:27.215967 25017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:14:27.216194 25017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:14:27.220031 25017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } }
I20260809 15:14:27.220597 25017 leader_election.cc:302] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 034df6000e114e9387fb2023002b04ba; no voters:
I20260809 15:14:27.221257 25017 leader_election.cc:288] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:14:27.221426 25021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:14:27.222790 25021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [term 2 LEADER]: Becoming Leader. State: Replica: 034df6000e114e9387fb2023002b04ba, State: Running, Role: LEADER
I20260809 15:14:27.223425 25021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } }
I20260809 15:14:27.223626 25017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:14:27.226258 25024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 034df6000e114e9387fb2023002b04ba. Latest consensus state: current_term: 2 leader_uuid: "034df6000e114e9387fb2023002b04ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } } }
I20260809 15:14:27.226704 25024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:27.227259 25023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "034df6000e114e9387fb2023002b04ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "034df6000e114e9387fb2023002b04ba" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35905 } } }
I20260809 15:14:27.227605 25023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:27.232499 25029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:14:27.239689 25029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:14:27.241447 25029 catalog_manager.cc:1259] Loaded cluster ID: 8ec5199d6036411890cd40cccc06f975
I20260809 15:14:27.241629 25029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:14:27.246888 25029 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:14:27.248804 25029 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 034df6000e114e9387fb2023002b04ba: Loaded TSK: 0
I20260809 15:14:27.250967 25029 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:14:27.541309 25019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:27.541625 25019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:27.541797 25019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:27.549010 25019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:27.549160 25019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:27.549321 25019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:14:27.558782 25019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.129.keytab
--principal=oryx/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:38011
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=34233
--tserver_master_addrs=127.15.11.190:35905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:27.560060 25019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:27.561403 25019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:27.575959 25045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:27.576525 25046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:27.576843 25048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:27.577410 25019 server_base.cc:1034] running on GCE node
I20260809 15:14:27.577988 25019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:27.578963 25019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:27.580232 25019 hybrid_clock.cc:648] HybridClock initialized: now 1786288467580172 us; error 51 us; skew 500 ppm
I20260809 15:14:27.580571 25019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:27 dist-test-slave-8w9v krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288467, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:27.588642 25019 init.cc:377] Logged in from keytab as oryx/127.15.11.129@KRBTEST.COM (short username oryx)
I20260809 15:14:27.590499 25019 webserver.cc:460] Webserver started at http://127.15.11.129:34233/ using document root <none> and password file <none>
I20260809 15:14:27.591536 25019 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:27.591717 25019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:27.597909 25019 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:14:27.600698 25055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:27.602137 25019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:14:27.602294 25019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "92d063391b224f1b8765df08025f1489"
format_stamp: "Formatted at 2026-08-09 15:14:24 on dist-test-slave-8w9v"
server_key: "d8c59e5cf6027392d56273a184d7fe7b"
server_key_iv: "9a30860b7751edf968aec91733d7dd03"
server_key_version: "encryptionkey@0"
I20260809 15:14:27.603263 25019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:27.625810 25019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:27.629964 25019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:27.630255 25019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:27.631268 25019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:27.632695 25019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:27.632828 25019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:27.632979 25019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:27.633066 25019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:27.666589 25019 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:38011
I20260809 15:14:27.666685 25182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:38011 every 8 connection(s)
I20260809 15:14:27.668105 25019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:14:27.677333 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 25019
I20260809 15:14:27.678051 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:34833
--local_ip_for_outbound_sockets=127.15.11.130
--tserver_master_addrs=127.15.11.190:35905
--webserver_port=33871
--webserver_interface=127.15.11.130
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.11.130
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:14:27 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:27 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:14:28.034951 25187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:28.035248 25187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:28.035437 25187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:28.042903 25187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:28.043063 25187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:28.043280 25187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:14:28.053329 25187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.130.keytab
--principal=oryx/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:34833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=33871
--tserver_master_addrs=127.15.11.190:35905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:28.054673 25187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:28.056162 25187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:28.074585 25194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:28.075977 25196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:28.076433 25193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:28.077759 25187 server_base.cc:1034] running on GCE node
I20260809 15:14:28.078456 25187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:28.079517 25187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:28.080742 25187 hybrid_clock.cc:648] HybridClock initialized: now 1786288468080695 us; error 37 us; skew 500 ppm
I20260809 15:14:28.081080 25187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:28 dist-test-slave-8w9v krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288468, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:28.084290 25187 init.cc:377] Logged in from keytab as oryx/127.15.11.130@KRBTEST.COM (short username oryx)
I20260809 15:14:28.085954 25187 webserver.cc:460] Webserver started at http://127.15.11.130:33871/ using document root <none> and password file <none>
I20260809 15:14:28.086673 25187 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:28.086825 25187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:28.093039 25187 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 15:14:28.096325 25204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:28.098003 25187 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:14:28.098208 25187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "9fca5d1db3ae43e9afbfd7af184c6e2c"
format_stamp: "Formatted at 2026-08-09 15:14:25 on dist-test-slave-8w9v"
server_key: "4bebf98ff436fc023d398e23e1d1ccee"
server_key_iv: "3159b00e85933c309051931bcb53494d"
server_key_version: "encryptionkey@0"
I20260809 15:14:28.099189 25187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:28.121451 25187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:28.126231 25187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:28.126518 25187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:28.127624 25187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:28.129181 25187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:28.129309 25187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:28.129467 25187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:28.129559 25187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:28.192950 25187 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:34833
I20260809 15:14:28.193354 25329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:34833 every 8 connection(s)
I20260809 15:14:28.194479 25187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:14:28.196671 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 25187
I20260809 15:14:28.197319 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:40641
--local_ip_for_outbound_sockets=127.15.11.131
--tserver_master_addrs=127.15.11.190:35905
--webserver_port=39211
--webserver_interface=127.15.11.131
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:37385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.11.131
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:14:28 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:28 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:14:28.870682 25334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:28.871279 25334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:28.871528 25334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:28.891026 25334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:28.891258 25334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:28.891575 25334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
W20260809 15:14:28.892830 25186 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.210s user 0.005s sys 0.011s
W20260809 15:14:28.895354 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:27.671137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:37319 (local address 127.15.11.190:35905)
0809 15:14:27.671922 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:27.671933 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:27.672199 (+ 266us) server_negotiation.cc:407] Connection header received
0809 15:14:27.674264 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:27.674275 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:27.674572 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:27.674846 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:27.676321 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:27.677727 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:27.680311 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:27.681081 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:27.681216 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:28.894204 (+1212988us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:37319: BlockingRecv error: recv got EOF from 127.15.11.129:37319 (error 108)
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":324,"threads_started":1}
I20260809 15:14:28.895351 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:27.671152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:37319)
0809 15:14:27.671982 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:27.672029 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:27.673786 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:27.674988 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:27.674995 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:27.675539 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:27.676085 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:27.676104 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:27.677910 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:27.677917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:27.680107 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:27.680148 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:27.681335 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:27.681344 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:27.681665 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:27.682413 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:27.682455 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:28.893781 (+1211326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":360,"threads_started":1}
W20260809 15:14:28.896755 25183 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:35905 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.11.190:35905: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Aug 09 15:14:28 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:28.908685 25334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:37385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/krb5kdc/oryx_127.15.11.131.keytab
--principal=oryx/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:40641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=39211
--tserver_master_addrs=127.15.11.190:35905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
Aug 09 15:14:28 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:28.910331 25334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:28.911391 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:28.897590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:54925)
0809 15:14:28.897897 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:28.897919 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:28.898053 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:28.898603 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:28.898613 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:28.899099 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:28.899343 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:28.899356 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.901352 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.901360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:28.902915 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:28.902935 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.904274 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.904326 (+ 52us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:28.904517 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:28.905372 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:28.905397 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:28.911070 (+ 5673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170}
W20260809 15:14:28.911851 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:28.897740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:54925 (local address 127.15.11.190:35905)
0809 15:14:28.897982 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:28.897991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:28.898023 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:14:28.898248 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:28.898256 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:28.898379 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:28.898536 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:28.899520 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.901167 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.903079 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.904038 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.904223 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:28.911352 (+ 7129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:54925: BlockingRecv error: recv got EOF from 127.15.11.129:54925 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 15:14:28.912318 25334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 09 15:14:28 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:28 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:14:28.934557 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:28.912575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:32897 (local address 127.15.11.190:35905)
0809 15:14:28.915581 (+ 3006us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:28.915590 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:28.918447 (+ 2857us) server_negotiation.cc:407] Connection header received
0809 15:14:28.918514 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:28.918519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:28.918630 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:28.918739 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:28.921145 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.922397 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.925331 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.926312 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.926445 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:28.934114 (+ 7669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:32897: BlockingRecv error: recv got EOF from 127.15.11.129:32897 (error 108)
Metrics: {"server-negotiator.queue_time_us":2859,"thread_start_us":9897,"threads_started":1}
I20260809 15:14:28.935775 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:28.912291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:32897)
0809 15:14:28.918144 (+ 5853us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:28.918165 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:28.918316 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:28.919406 (+ 1090us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:28.919411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:28.919829 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:28.920052 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:28.920065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.922713 (+ 2648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.922722 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:28.924976 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:28.924993 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.926831 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.926841 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:28.927163 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:28.928095 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:28.928170 (+ 75us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:28.935127 (+ 6957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":5737,"thread_start_us":11836,"threads_started":1}
W20260809 15:14:28.936506 25183 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:14:28.937055 25343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:28.939352 25344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:28.939993 25347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:28.941072 25334 server_base.cc:1034] running on GCE node
I20260809 15:14:28.941793 25334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:28.942862 25334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:28.944205 25334 hybrid_clock.cc:648] HybridClock initialized: now 1786288468944111 us; error 79 us; skew 500 ppm
I20260809 15:14:28.944561 25334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:28 dist-test-slave-8w9v krb5kdc[24362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288468, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:28.948261 25334 init.cc:377] Logged in from keytab as oryx/127.15.11.131@KRBTEST.COM (short username oryx)
I20260809 15:14:28.949924 25334 webserver.cc:460] Webserver started at http://127.15.11.131:39211/ using document root <none> and password file <none>
I20260809 15:14:28.950619 25334 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:28.950757 25334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:28.956950 25334 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:14:28.961647 25354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:28.963841 25334 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260809 15:14:28.964131 25334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "14190964039f40a092aa0a276701e5e9"
format_stamp: "Formatted at 2026-08-09 15:14:25 on dist-test-slave-8w9v"
server_key: "50005c1b01f1d710a271c045820764b3"
server_key_iv: "e8bb63f784c9dfb9922536796b848805"
server_key_version: "encryptionkey@0"
I20260809 15:14:28.966238 25334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:29.007181 25334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:29.015461 25334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:29.015969 25334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:29.017887 25334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:29.020656 25334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:29.020872 25334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:29.021080 25334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:29.021219 25334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:29.065042 25334 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:40641
I20260809 15:14:29.064999 25479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:40641 every 8 connection(s)
I20260809 15:14:29.068034 25334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:14:29.077652 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 25334
Aug 09 15:14:29 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:29 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:29.094487 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:29.078605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46784 (local address 127.15.11.190:35905)
0809 15:14:29.078838 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:29.078847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:29.079407 (+ 560us) server_negotiation.cc:407] Connection header received
0809 15:14:29.079592 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:29.079598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:29.079665 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:29.079776 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:29.081467 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.082279 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.083926 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.084429 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.084567 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:29.087501 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:29.087510 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:29.087528 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:29.087564 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:29.091127 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:29.091827 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:29.091837 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:29.091853 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:29.092029 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:29.092458 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:29.092465 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:29.092468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:29.092953 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:29.093292 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:29.093769 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 15:14:29.094191 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 15:14:29.446426 25333 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.218s user 0.005s sys 0.024s
W20260809 15:14:29.447938 25332 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:28.198500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:59529 (local address 127.15.11.190:35905)
0809 15:14:28.213729 (+ 15229us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:28.213737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:28.213764 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:14:28.216869 (+ 3105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:28.216879 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:28.217010 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:28.217151 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:28.220348 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.221687 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.225761 (+ 4074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.226534 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.226698 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:29.447526 (+1220828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:59529: BlockingRecv error: recv got EOF from 127.15.11.130:59529 (error 108)
Metrics: {"server-negotiator.queue_time_us":15068,"thread_start_us":15265,"threads_started":1}
I20260809 15:14:29.448571 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:28.199656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:59529)
0809 15:14:28.212083 (+ 12427us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:28.212190 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:28.216392 (+ 4202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:28.217612 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:28.217632 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:28.218820 (+ 1188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:28.220037 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:28.220075 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.221924 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.221934 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:28.225542 (+ 3608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:28.225568 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:28.226917 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:28.226930 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:28.227440 (+ 510us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:28.228618 (+ 1178us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:28.228681 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:29.447337 (+1218656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":10625,"thread_start_us":10576,"threads_started":1}
W20260809 15:14:29.449606 25330 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:35905 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.11.190:35905: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Aug 09 15:14:29 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:29 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:29.460661 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:29.450479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:47041)
0809 15:14:29.450729 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:29.450767 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:29.450919 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:29.451268 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:29.451272 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:29.451670 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:29.451856 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:29.451865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.453024 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.453029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:29.453949 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:29.453960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.454721 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.454728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:29.454903 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:29.455727 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:29.455755 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:29.460335 (+ 4580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109}
W20260809 15:14:29.460812 25332 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:29.450483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:47041 (local address 127.15.11.190:35905)
0809 15:14:29.450668 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:29.450675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:29.450840 (+ 165us) server_negotiation.cc:407] Connection header received
0809 15:14:29.451061 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:29.451066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:29.451130 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:29.451244 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:29.452009 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.452866 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.454103 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.454510 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.454662 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:29.460544 (+ 5882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:47041: BlockingRecv error: recv got EOF from 127.15.11.130:47041 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 15:14:29 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:29 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:29.473706 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:29.461658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:60611)
0809 15:14:29.461895 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:29.461922 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:29.462060 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:29.462428 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:29.462436 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:29.463027 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:29.463315 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:29.463330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.464502 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.464511 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:29.466704 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:29.466724 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.467507 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.467516 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:29.467812 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:29.468590 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:29.468624 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:29.473252 (+ 4628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 15:14:29.473749 25332 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:29.461833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:60611 (local address 127.15.11.190:35905)
0809 15:14:29.461976 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:29.461980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:29.461997 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:14:29.462190 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:29.462193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:29.462250 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:29.462329 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:29.463523 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.464335 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.466872 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.467319 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.467417 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:29.473410 (+ 5993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:60611: BlockingRecv error: recv got EOF from 127.15.11.130:60611 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 15:14:29.474224 25330 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 15:14:29 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:29 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:29.948578 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:29.937289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:55311)
0809 15:14:29.937884 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:29.937902 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:29.938014 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:29.938514 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:29.938517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:29.938962 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:29.939112 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:29.939121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.941349 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.941358 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:29.942574 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:29.942590 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.943461 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.943468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:29.943654 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:29.944346 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:29.944366 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:29.948224 (+ 3858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":350,"threads_started":1}
W20260809 15:14:29.948913 25332 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:29.937646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:55311 (local address 127.15.11.190:35905)
0809 15:14:29.937949 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:29.937960 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:29.937998 (+ 38us) server_negotiation.cc:407] Connection header received
0809 15:14:29.938150 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:29.938159 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:29.938319 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:29.938510 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:29.939300 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.941135 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.942788 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.943265 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.943425 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:29.948447 (+ 5022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:55311: BlockingRecv error: recv got EOF from 127.15.11.129:55311 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
W20260809 15:14:30.177307 25483 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.092s user 0.012s sys 0.018s
W20260809 15:14:30.178399 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:29.071374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:53873 (local address 127.15.11.190:35905)
0809 15:14:29.071712 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:29.071724 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:29.073031 (+ 1307us) server_negotiation.cc:407] Connection header received
0809 15:14:29.075173 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:29.075183 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:29.075316 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:29.075540 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:29.077211 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.078960 (+ 1749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.082900 (+ 3940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.083546 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.083687 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:30.178188 (+1094501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:53873: BlockingRecv error: recv got EOF from 127.15.11.131:53873 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
I20260809 15:14:30.179008 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:29.071877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:53873)
0809 15:14:29.072829 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:29.072901 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:29.074881 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:29.075734 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:29.075742 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:29.076394 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:29.076977 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:29.076996 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.079306 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.079317 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:29.082656 (+ 3339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:29.082685 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:29.083921 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:29.083934 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:29.084466 (+ 532us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:29.085601 (+ 1135us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:29.085665 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:30.178030 (+1092365us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":455,"threads_started":1}
W20260809 15:14:30.179953 25480 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:35905 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.11.190:35905: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Aug 09 15:14:30 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:30 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:30.190039 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:30.180716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:50319)
0809 15:14:30.180922 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:30.180942 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:30.181074 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:30.181516 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:30.181539 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:30.182039 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:30.182238 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:30.182247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.183758 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.183763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:30.184803 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:30.184816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.185484 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.185488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:30.185600 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:30.186148 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:30.186165 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:30.189751 (+ 3586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 15:14:30.190241 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:30.180946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:50319 (local address 127.15.11.190:35905)
0809 15:14:30.181222 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:30.181227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:30.181250 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:14:30.181292 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:30.181295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:30.181351 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:30.181471 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:30.182432 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.183619 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.184924 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.185331 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.185431 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:30.189978 (+ 4547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:50319: BlockingRecv error: recv got EOF from 127.15.11.131:50319 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Aug 09 15:14:30 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:30 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:30.198734 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:30.190937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:34083)
0809 15:14:30.191080 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:30.191100 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:30.191192 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:30.191519 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:30.191524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:30.191821 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:30.191957 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:30.191964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.192959 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.192970 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:30.193931 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:30.193943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.194630 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.194636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:30.194830 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:30.195329 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:30.195345 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:30.198447 (+ 3102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 15:14:30.198910 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:30.191012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:34083 (local address 127.15.11.190:35905)
0809 15:14:30.191193 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:30.191199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:30.191217 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:14:30.191326 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:30.191331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:30.191396 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:30.191493 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:30.192089 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.192820 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.194070 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.194483 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.194611 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:30.198670 (+ 4059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:34083: BlockingRecv error: recv got EOF from 127.15.11.131:34083 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 15:14:30.199043 25480 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 15:14:30 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:30 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:30.484081 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:30.474798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:41875)
0809 15:14:30.475438 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:30.475461 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:30.475604 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:30.475965 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:30.475969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:30.476497 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:30.476715 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:30.476725 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.477699 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.477704 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:30.478693 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:30.478704 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.479420 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.479426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:30.479606 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:30.480245 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:30.480263 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:30.483695 (+ 3432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":323,"threads_started":1}
W20260809 15:14:30.484155 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:30.474897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:41875 (local address 127.15.11.190:35905)
0809 15:14:30.475108 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:30.475117 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:30.475533 (+ 416us) server_negotiation.cc:407] Connection header received
0809 15:14:30.475744 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:30.475749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:30.475818 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:30.475922 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:30.476851 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.477558 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.478835 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.479234 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.479382 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:30.483892 (+ 4510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:41875: BlockingRecv error: recv got EOF from 127.15.11.130:41875 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 15:14:30 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:30 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:30.958606 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:30.949495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:53379)
0809 15:14:30.949942 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:30.949959 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:30.950105 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:30.950379 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:30.950382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:30.950830 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:30.951015 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:30.951024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.951991 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.951994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:30.953020 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:30.953033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.953719 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.953723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:30.953869 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:30.954429 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:30.954447 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:30.958242 (+ 3795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":230,"threads_started":1}
W20260809 15:14:30.958709 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:30.949576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:53379 (local address 127.15.11.190:35905)
0809 15:14:30.949814 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:30.949820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:30.950057 (+ 237us) server_negotiation.cc:407] Connection header received
0809 15:14:30.950198 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:30.950201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:30.950258 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:30.950332 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:30.951140 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.951853 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.953147 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:30.953579 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:30.953678 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:30.958448 (+ 4770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:53379: BlockingRecv error: recv got EOF from 127.15.11.129:53379 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 09 15:14:31 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:31 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:31.209030 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:31.199695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:39551)
0809 15:14:31.200250 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:31.200271 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:31.200433 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:31.200761 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:31.200767 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:31.201253 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:31.201406 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:31.201415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.202486 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.202495 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:31.203543 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:31.203557 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.204358 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.204365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:31.204540 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:31.205073 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:31.205090 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:31.208792 (+ 3702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":273,"threads_started":1}
W20260809 15:14:31.209188 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:31.199741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:39551 (local address 127.15.11.190:35905)
0809 15:14:31.199937 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:31.199945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:31.200351 (+ 406us) server_negotiation.cc:407] Connection header received
0809 15:14:31.200547 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:31.200558 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:31.200628 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:31.200724 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:31.201546 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.202320 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.203687 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.204182 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.204315 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:31.208980 (+ 4665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:39551: BlockingRecv error: recv got EOF from 127.15.11.131:39551 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 15:14:31 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:31 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:31.494948 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:31.485311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:59517)
0809 15:14:31.485855 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:31.485884 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:31.486047 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:31.486407 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:31.486412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:31.486908 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:31.487101 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:31.487110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.488200 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.488209 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:31.489236 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:31.489251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.490120 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.490129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:31.490369 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:31.491139 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:31.491167 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:31.494696 (+ 3529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":333,"threads_started":1}
W20260809 15:14:31.495146 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:31.485381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:59517 (local address 127.15.11.190:35905)
0809 15:14:31.485634 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:31.485642 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:31.485971 (+ 329us) server_negotiation.cc:407] Connection header received
0809 15:14:31.486169 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:31.486174 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:31.486242 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:31.486348 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:31.487248 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.488019 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.489395 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.489943 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.490056 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:31.494896 (+ 4840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:59517: BlockingRecv error: recv got EOF from 127.15.11.130:59517 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 15:14:31 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:31 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:31.974244 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:31.959793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:41713)
0809 15:14:31.960378 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:31.960395 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:31.960511 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:31.960795 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:31.960799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:31.961181 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:31.961356 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:31.961365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.962298 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.962301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:31.963168 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:31.963179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.964227 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.964233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:31.964415 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:31.970324 (+ 5909us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:31.970342 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:31.973977 (+ 3635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":299,"threads_started":1}
W20260809 15:14:31.974318 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:31.960096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:41713 (local address 127.15.11.190:35905)
0809 15:14:31.960313 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:31.960317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:31.960470 (+ 153us) server_negotiation.cc:407] Connection header received
0809 15:14:31.960604 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:31.960607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:31.960667 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:31.960746 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:31.961490 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.962187 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.963298 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:31.964005 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:31.964155 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:31.974131 (+ 9976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:41713: BlockingRecv error: recv got EOF from 127.15.11.129:41713 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 09 15:14:32 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:32 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:32.218705 25492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:32.210151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:45303)
0809 15:14:32.210615 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:32.210635 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:32.210747 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:32.211070 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:32.211073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:32.211557 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:32.211740 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:32.211750 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.212913 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.212918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:32.213972 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:32.213984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.214595 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.214599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:32.214709 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:32.215303 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:32.215321 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:32.218472 (+ 3151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":266,"threads_started":1}
W20260809 15:14:32.218866 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:32.210240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:45303 (local address 127.15.11.190:35905)
0809 15:14:32.210499 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:32.210506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:32.210715 (+ 209us) server_negotiation.cc:407] Connection header received
0809 15:14:32.210843 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:32.210847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:32.210921 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:32.211014 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:32.211872 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.212767 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.214090 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.214475 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.214562 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:32.218645 (+ 4083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:45303: BlockingRecv error: recv got EOF from 127.15.11.131:45303 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 09 15:14:32 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:32 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:32.505192 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:32.495898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:43195)
0809 15:14:32.496493 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:32.496509 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:32.496623 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:32.496901 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:32.496904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:32.497284 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:32.497545 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:32.497560 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.498941 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.498945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:32.499814 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:32.499825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.500527 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.500532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:32.500656 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:32.501367 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:32.501385 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:32.504838 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":346,"threads_started":1}
W20260809 15:14:32.505239 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:32.495937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:43195 (local address 127.15.11.190:35905)
0809 15:14:32.496199 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:32.496208 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:32.496589 (+ 381us) server_negotiation.cc:407] Connection header received
0809 15:14:32.496716 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:32.496720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:32.496780 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:32.496858 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:32.497714 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.498808 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.499926 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.500397 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.500496 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:32.505052 (+ 4556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:43195: BlockingRecv error: recv got EOF from 127.15.11.130:43195 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 15:14:32 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:32 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:32.989524 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:32.975341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:42393)
0809 15:14:32.975888 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:32.975905 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:32.976033 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:32.976371 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:32.976374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:32.976825 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:32.977002 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:32.977011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.978086 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.978091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:32.979107 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:32.979119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.979839 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.979843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:32.979978 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:32.985689 (+ 5711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:32.985709 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:32.989257 (+ 3548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":310,"threads_started":1}
W20260809 15:14:32.989704 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:32.975395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:42393 (local address 127.15.11.190:35905)
0809 15:14:32.975636 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:32.975646 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:32.975985 (+ 339us) server_negotiation.cc:407] Connection header received
0809 15:14:32.976154 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:32.976158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:32.976233 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:32.976312 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:32.977183 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.977950 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.979265 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:32.979689 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:32.979796 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:32.989464 (+ 9668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:42393: BlockingRecv error: recv got EOF from 127.15.11.129:42393 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 15:14:33 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:33 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:33.228482 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:33.219773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:42897)
0809 15:14:33.220324 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:33.220345 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:33.220458 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:33.220735 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:33.220738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:33.221111 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:33.221291 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:33.221299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.222246 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.222250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:33.223145 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:33.223156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.223850 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.223854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:33.223986 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:33.224719 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:33.224747 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:33.228105 (+ 3358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":282,"threads_started":1}
W20260809 15:14:33.228663 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:33.219829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:42897 (local address 127.15.11.190:35905)
0809 15:14:33.220054 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:33.220063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:33.220424 (+ 361us) server_negotiation.cc:407] Connection header received
0809 15:14:33.220537 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:33.220541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:33.220613 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:33.220702 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:33.221419 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.222126 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.223260 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.223726 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.223816 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:33.228324 (+ 4508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:42897: BlockingRecv error: recv got EOF from 127.15.11.131:42897 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 15:14:33 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:33 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:33.515234 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:33.506191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:50655)
0809 15:14:33.506751 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:33.506766 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:33.506883 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:33.507280 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:33.507284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:33.507662 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:33.507843 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:33.507850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.508905 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.508908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:33.509933 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:33.509943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.510604 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.510608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:33.510735 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:33.511306 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:33.511323 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:33.514991 (+ 3668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":316,"threads_started":1}
W20260809 15:14:33.515458 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:33.506250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:50655 (local address 127.15.11.190:35905)
0809 15:14:33.506409 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:33.506414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:33.506847 (+ 433us) server_negotiation.cc:407] Connection header received
0809 15:14:33.507020 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:33.507025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:33.507121 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:33.507231 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:33.507980 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.508767 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.510075 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.510464 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.510598 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:33.515181 (+ 4583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:50655: BlockingRecv error: recv got EOF from 127.15.11.130:50655 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 15:14:33 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:33 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:33.999392 25498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:33.990537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:58417)
0809 15:14:33.991112 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:33.991130 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:33.991245 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:33.991558 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:33.991562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:33.991980 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:33.992208 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:33.992220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.993211 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.993214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:33.994113 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:33.994124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.994782 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.994786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:33.994911 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:33.995541 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:33.995559 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:33.999148 (+ 3589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":322,"threads_started":1}
W20260809 15:14:33.999545 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:33.990655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:58417 (local address 127.15.11.190:35905)
0809 15:14:33.990830 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:33.990836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:33.991204 (+ 368us) server_negotiation.cc:407] Connection header received
0809 15:14:33.991347 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:33.991351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:33.991423 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:33.991505 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:33.992339 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.993094 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.994233 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:33.994660 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:33.994747 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:33.999315 (+ 4568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:58417: BlockingRecv error: recv got EOF from 127.15.11.129:58417 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 15:14:34 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:34 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:34.238732 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:34.229621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:35865)
0809 15:14:34.230162 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:34.230200 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:34.230318 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:34.230635 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:34.230638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:34.231012 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:34.231178 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:34.231186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:34.232194 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:34.232198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:34.233326 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:34.233347 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:34.234062 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:34.234067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:34.234192 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:34.234792 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:34.234809 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:34.238488 (+ 3679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":297,"threads_started":1}
W20260809 15:14:34.238883 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:34.229715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:35865 (local address 127.15.11.190:35905)
0809 15:14:34.229879 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:34.229888 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:34.230281 (+ 393us) server_negotiation.cc:407] Connection header received
0809 15:14:34.230443 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:34.230447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:34.230506 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:34.230587 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:34.231301 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:34.232008 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:34.233460 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:34.233937 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:34.234034 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:34.238670 (+ 4636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:35865: BlockingRecv error: recv got EOF from 127.15.11.131:35865 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 15:14:34 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:34 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:34.525444 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:34.516293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:35553)
0809 15:14:34.516813 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:34.516828 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:34.516976 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:34.517302 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:34.517306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:34.517902 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:34.518150 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:34.518161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:34.519237 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:34.519242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:34.520191 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:34.520202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:34.520895 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:34.520900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:34.521068 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:34.521716 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:34.521744 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:34.525219 (+ 3475us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":261,"threads_started":1}
W20260809 15:14:34.525641 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:34.516465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:35553 (local address 127.15.11.190:35905)
0809 15:14:34.516678 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:34.516684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:34.516910 (+ 226us) server_negotiation.cc:407] Connection header received
0809 15:14:34.517108 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:34.517113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:34.517172 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:34.517271 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:34.518355 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:34.519100 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:34.520327 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:34.520748 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:34.520910 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:34.525432 (+ 4522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:35553: BlockingRecv error: recv got EOF from 127.15.11.130:35553 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 15:14:35 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:35 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:35.009999 25502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:35.000489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:41861)
0809 15:14:35.001018 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:35.001033 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:35.001192 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:35.001520 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:35.001526 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:35.001961 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:35.002128 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:35.002137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.003243 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.003254 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:35.004400 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:35.004414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.005142 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.005147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:35.005331 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:35.005945 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:35.005963 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:35.009692 (+ 3729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":299,"threads_started":1}
W20260809 15:14:35.010128 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:35.000546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:41861 (local address 127.15.11.190:35905)
0809 15:14:35.000789 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:35.000797 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:35.001118 (+ 321us) server_negotiation.cc:407] Connection header received
0809 15:14:35.001315 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:35.001324 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:35.001383 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:35.001484 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:35.002266 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.003093 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.004541 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.004976 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.005117 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:35.009922 (+ 4805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:41861: BlockingRecv error: recv got EOF from 127.15.11.129:41861 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 15:14:35 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:35 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:35.249394 25503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:35.239948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:37085)
0809 15:14:35.240804 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:35.240826 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:35.241007 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:35.241366 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:35.241374 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:35.241885 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:35.242046 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:35.242055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.243135 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.243141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:35.244104 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:35.244144 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.244903 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.244910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:35.245091 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:35.245672 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:35.245690 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:35.249059 (+ 3369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":727,"thread_start_us":593,"threads_started":1}
W20260809 15:14:35.249531 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:35.240254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:37085 (local address 127.15.11.190:35905)
0809 15:14:35.240486 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:35.240508 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:35.240926 (+ 418us) server_negotiation.cc:407] Connection header received
0809 15:14:35.241141 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:35.241146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:35.241218 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:35.241323 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:35.242201 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.242982 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.244283 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.244744 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.244880 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:35.249281 (+ 4401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:37085: BlockingRecv error: recv got EOF from 127.15.11.131:37085 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 15:14:35 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:35 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:35.535784 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:35.526491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:44461)
0809 15:14:35.527006 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:35.527033 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:35.527146 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:35.527462 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:35.527466 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:35.527897 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:35.528066 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:35.528074 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.529138 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.529141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:35.530072 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:35.530082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.530699 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.530703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:35.530818 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:35.531440 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:35.531457 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:35.535447 (+ 3990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":294,"threads_started":1}
W20260809 15:14:35.535854 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:35.526566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:44461 (local address 127.15.11.190:35905)
0809 15:14:35.526761 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:35.526765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:35.527107 (+ 342us) server_negotiation.cc:407] Connection header received
0809 15:14:35.527239 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:35.527243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:35.527301 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:35.527380 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:35.528272 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.528991 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.530189 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:35.530574 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:35.530655 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:35.535639 (+ 4984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:44461: BlockingRecv error: recv got EOF from 127.15.11.130:44461 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 15:14:36 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:36 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:36.019615 25506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:36.010888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:46735)
0809 15:14:36.011534 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:36.011550 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:36.011688 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:36.011989 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:36.011992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:36.012407 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:36.012581 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:36.012589 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.013524 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.013527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:36.014439 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:36.014450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.015053 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.015057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:36.015163 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:36.015767 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:36.015784 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:36.019239 (+ 3455us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":283,"threads_started":1}
W20260809 15:14:36.019727 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:36.010901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:46735 (local address 127.15.11.190:35905)
0809 15:14:36.011091 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:36.011095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:36.011640 (+ 545us) server_negotiation.cc:407] Connection header received
0809 15:14:36.011784 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:36.011788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:36.011848 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:36.011928 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:36.012697 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.013403 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.014552 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.014934 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.015015 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:36.019484 (+ 4469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:46735: BlockingRecv error: recv got EOF from 127.15.11.129:46735 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 15:14:36 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:36 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:14:36.260412 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:36.250634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:41779 (local address 127.15.11.190:35905)
0809 15:14:36.250808 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:36.250812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:36.251269 (+ 457us) server_negotiation.cc:407] Connection header received
0809 15:14:36.251422 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:36.251425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:36.251484 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:36.251561 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:36.252471 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.253181 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.254469 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.254876 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.254972 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:36.260094 (+ 5122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:41779: BlockingRecv error: recv got EOF from 127.15.11.131:41779 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:14:36.260481 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:36.250586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:41779)
0809 15:14:36.251163 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:36.251189 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:36.251323 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:36.251619 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:36.251622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:36.252081 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:36.252301 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:36.252329 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.253345 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.253350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:36.254334 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:36.254347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.255030 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.255035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:36.255169 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:36.255853 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:36.255873 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:36.259882 (+ 4009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":325,"threads_started":1}
Aug 09 15:14:36 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:36 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:36.545804 25508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:36.536849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:36647)
0809 15:14:36.537496 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:36.537513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:36.537629 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:36.537921 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:36.537924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:36.538335 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:36.538529 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:36.538537 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.539476 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.539480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:36.540520 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:36.540532 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.541216 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.541220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:36.541328 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:36.541905 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:36.541931 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:36.545568 (+ 3637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":297,"threads_started":1}
W20260809 15:14:36.545997 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:36.536915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:36647 (local address 127.15.11.190:35905)
0809 15:14:36.537224 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:36.537233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:36.537598 (+ 365us) server_negotiation.cc:407] Connection header received
0809 15:14:36.537728 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:36.537731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:36.537794 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:36.537870 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:36.538652 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.539356 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.540642 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:36.541078 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:36.541179 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:36.545747 (+ 4568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:36647: BlockingRecv error: recv got EOF from 127.15.11.130:36647 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 09 15:14:37 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:37 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:37.030275 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:37.020707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:57635)
0809 15:14:37.021293 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:37.021325 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:37.021488 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:37.021855 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:37.021859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:37.022343 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:37.022533 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:37.022542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.023605 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.023613 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:37.024704 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:37.024716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.025409 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.025415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:37.025600 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:37.026173 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:37.026189 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:37.029924 (+ 3735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":323,"threads_started":1}
W20260809 15:14:37.030432 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:37.020800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:57635 (local address 127.15.11.190:35905)
0809 15:14:37.021045 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:37.021065 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:37.021420 (+ 355us) server_negotiation.cc:407] Connection header received
0809 15:14:37.021609 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:37.021616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:37.021683 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:37.021804 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:37.022678 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.023476 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.024832 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.025235 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.025364 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:37.030170 (+ 4806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:57635: BlockingRecv error: recv got EOF from 127.15.11.129:57635 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 15:14:37 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:37 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:37.270834 25511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:37.261434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:36615)
0809 15:14:37.262114 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:37.262139 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:37.262261 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:37.262578 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:37.262583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:37.263027 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:37.263217 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:37.263226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.264251 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.264255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:37.265326 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:37.265339 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.266240 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.266246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:37.266383 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:37.266963 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:37.266981 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:37.270476 (+ 3495us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":343,"threads_started":1}
W20260809 15:14:37.270923 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:37.261707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:36615 (local address 127.15.11.190:35905)
0809 15:14:37.261893 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:37.261898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:37.262218 (+ 320us) server_negotiation.cc:407] Connection header received
0809 15:14:37.262347 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:37.262350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:37.262419 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:37.262501 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:37.263345 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.264049 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.265455 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.266097 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.266193 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:37.270670 (+ 4477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:36615: BlockingRecv error: recv got EOF from 127.15.11.131:36615 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 15:14:37 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:37 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:37.555471 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:37.546723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:58103)
0809 15:14:37.547258 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:37.547275 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:37.547391 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:37.547677 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:37.547679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:37.548094 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:37.548286 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:37.548295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.549341 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.549344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:37.550454 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:37.550466 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.551311 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.551316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:37.551435 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:37.551978 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:37.551994 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:37.555239 (+ 3245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":277,"threads_started":1}
W20260809 15:14:37.555637 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:37.546926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:58103 (local address 127.15.11.190:35905)
0809 15:14:37.547128 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:37.547133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:37.547349 (+ 216us) server_negotiation.cc:407] Connection header received
0809 15:14:37.547487 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:37.547490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:37.547549 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:37.547628 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:37.548414 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.549229 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.550618 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:37.551153 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:37.551244 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:37.555413 (+ 4169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:58103: BlockingRecv error: recv got EOF from 127.15.11.130:58103 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 15:14:38 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:38 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:38.045234 25513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:38.031251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:50379)
0809 15:14:38.031845 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:38.031870 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:38.031985 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:38.032290 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:38.032294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:38.032772 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:38.032980 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:38.032988 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.034077 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.034081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:38.035062 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:38.035075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.036281 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.036288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:38.036496 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:38.041358 (+ 4862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:38.041379 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:38.044989 (+ 3610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":347,"threads_started":1}
W20260809 15:14:38.045383 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:38.031333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:50379 (local address 127.15.11.190:35905)
0809 15:14:38.031517 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:38.031521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:38.031945 (+ 424us) server_negotiation.cc:407] Connection header received
0809 15:14:38.032082 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:38.032086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:38.032164 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:38.032239 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:38.033100 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.033932 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.035406 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.036085 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.036237 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:38.045171 (+ 8934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:50379: BlockingRecv error: recv got EOF from 127.15.11.129:50379 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 15:14:38 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:38 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:38.281200 25514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:38.272090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:49605)
0809 15:14:38.272693 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:38.272718 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:38.272860 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:38.273260 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:38.273268 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:38.273882 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:38.274075 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:38.274084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.275069 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.275073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:38.276017 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:38.276028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.276772 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.276777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:38.276957 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:38.277649 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:38.277666 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:38.280958 (+ 3292us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"spinlock_wait_cycles":30464,"thread_start_us":380,"threads_started":1}
W20260809 15:14:38.281330 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:38.272210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:49605 (local address 127.15.11.190:35905)
0809 15:14:38.272430 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:38.272434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:38.272809 (+ 375us) server_negotiation.cc:407] Connection header received
0809 15:14:38.272986 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:38.272992 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:38.273069 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:38.273186 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:38.274195 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.274928 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.276153 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.276618 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.276711 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:38.281141 (+ 4430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:49605: BlockingRecv error: recv got EOF from 127.15.11.131:49605 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 15:14:38 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:38 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:38.566356 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:38.556654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:55521)
0809 15:14:38.557387 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:38.557412 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:38.557581 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:38.558012 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:38.558016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:38.558505 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:38.558690 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:38.558699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.559749 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.559754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:38.560764 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:38.560775 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.561533 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.561538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:38.561711 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:38.562388 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:38.562406 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:38.566042 (+ 3636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":400,"threads_started":1}
W20260809 15:14:38.566491 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:38.556698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:55521 (local address 127.15.11.190:35905)
0809 15:14:38.556948 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:38.556957 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:38.557509 (+ 552us) server_negotiation.cc:407] Connection header received
0809 15:14:38.557768 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:38.557772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:38.557852 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:38.557960 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:38.558848 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.559604 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.560909 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:38.561374 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:38.561501 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:38.566240 (+ 4739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:55521: BlockingRecv error: recv got EOF from 127.15.11.130:55521 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 15:14:39 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:39 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:39.055328 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:39.046217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:37839)
0809 15:14:39.046755 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:39.046771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:39.046966 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:39.047323 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:39.047328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:39.047768 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:39.047949 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:39.047959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.049003 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.049011 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:39.049984 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:39.049995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.050721 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.050728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:39.050898 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:39.051458 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:39.051476 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:39.055091 (+ 3615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":237,"threads_started":1}
W20260809 15:14:39.055531 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:39.046463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:37839 (local address 127.15.11.190:35905)
0809 15:14:39.046685 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:39.046691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:39.046863 (+ 172us) server_negotiation.cc:407] Connection header received
0809 15:14:39.047090 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:39.047095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:39.047165 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:39.047266 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:39.048103 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.048857 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.050132 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.050541 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.050680 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:39.055270 (+ 4590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:37839: BlockingRecv error: recv got EOF from 127.15.11.129:37839 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 15:14:39 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:39 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:39.291473 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:39.282300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:41195)
0809 15:14:39.282870 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:39.282894 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:39.283045 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:39.283345 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:39.283349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:39.283796 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:39.283958 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:39.283967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.284947 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.284951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:39.285917 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:39.285931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.286595 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.286600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:39.286754 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:39.287347 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:39.287366 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:39.291105 (+ 3739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":310,"threads_started":1}
W20260809 15:14:39.291584 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:39.282327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:41195 (local address 127.15.11.190:35905)
0809 15:14:39.282516 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:39.282520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:39.282994 (+ 474us) server_negotiation.cc:407] Connection header received
0809 15:14:39.283147 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:39.283151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:39.283207 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:39.283282 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:39.284091 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.284813 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.286053 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.286451 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.286540 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:39.291299 (+ 4759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:41195: BlockingRecv error: recv got EOF from 127.15.11.131:41195 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 15:14:39 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:39 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:39.577033 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:39.567622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:60861)
0809 15:14:39.568243 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:39.568259 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:39.568430 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:39.568794 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:39.568799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:39.569251 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:39.569423 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:39.569431 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.570382 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.570390 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:39.571366 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:39.571378 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.572097 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.572103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:39.572300 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:39.572979 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:39.573003 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:39.576669 (+ 3666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":275,"threads_started":1}
W20260809 15:14:39.577095 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:39.567697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:60861 (local address 127.15.11.190:35905)
0809 15:14:39.567931 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:39.567939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:39.568378 (+ 439us) server_negotiation.cc:407] Connection header received
0809 15:14:39.568589 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:39.568593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:39.568653 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:39.568757 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:39.569571 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.570248 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.571507 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:39.571946 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:39.572089 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:39.576865 (+ 4776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:60861: BlockingRecv error: recv got EOF from 127.15.11.130:60861 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 15:14:40 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:40 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:40.064821 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:40.056433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:48981)
0809 15:14:40.056914 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:40.056930 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:40.057052 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:40.057312 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:40.057314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:40.057734 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:40.057918 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:40.057927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.058851 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.058854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:40.059731 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:40.059741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.060409 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.060413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:40.060539 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:40.061085 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:40.061101 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:40.064460 (+ 3359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":261,"threads_started":1}
W20260809 15:14:40.064906 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:40.056522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:48981 (local address 127.15.11.190:35905)
0809 15:14:40.056750 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:40.056756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:40.057018 (+ 262us) server_negotiation.cc:407] Connection header received
0809 15:14:40.057155 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:40.057158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:40.057221 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:40.057298 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:40.058034 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.058737 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.059840 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.060281 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.060369 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:40.064648 (+ 4279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:48981: BlockingRecv error: recv got EOF from 127.15.11.129:48981 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 15:14:40 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:40 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:40.301007 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:40.292433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:51469)
0809 15:14:40.292979 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:40.293000 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:40.293110 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:40.293427 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:40.293430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:40.293792 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:40.293954 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:40.293962 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.294914 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.294917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:40.295909 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:40.295921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.296661 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.296666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:40.296779 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:40.297328 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:40.297345 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:40.300761 (+ 3416us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":296,"threads_started":1}
W20260809 15:14:40.301144 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:40.292473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:51469 (local address 127.15.11.190:35905)
0809 15:14:40.292642 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:40.292646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:40.293070 (+ 424us) server_negotiation.cc:407] Connection header received
0809 15:14:40.293200 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:40.293204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:40.293293 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:40.293380 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:40.294072 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.294779 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.296049 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.296531 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.296624 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:40.300929 (+ 4305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:51469: BlockingRecv error: recv got EOF from 127.15.11.131:51469 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 15:14:40 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:40 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:40.587008 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:40.578112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:54907)
0809 15:14:40.578644 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:40.578659 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:40.578809 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:40.579165 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:40.579169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:40.579655 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:40.579827 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:40.579836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.580888 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.580893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:40.581721 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:40.581731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.582452 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.582457 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:40.582619 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:40.583212 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:40.583229 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:40.586678 (+ 3449us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":290,"threads_started":1}
W20260809 15:14:40.587127 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:40.578195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:54907 (local address 127.15.11.190:35905)
0809 15:14:40.578448 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:40.578457 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:40.578743 (+ 286us) server_negotiation.cc:407] Connection header received
0809 15:14:40.578939 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:40.578945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:40.579027 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:40.579127 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:40.579986 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.580763 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.581854 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:40.582292 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:40.582414 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:40.586849 (+ 4435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:54907: BlockingRecv error: recv got EOF from 127.15.11.130:54907 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 09 15:14:41 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:41 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:41.075340 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:41.065909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:51383)
0809 15:14:41.066528 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:41.066545 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:41.066660 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:41.067020 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:41.067023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:41.067398 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:41.067584 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:41.067592 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.068982 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.068987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:41.070052 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:41.070066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.070721 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.070725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:41.070851 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:41.071430 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:41.071446 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:41.075112 (+ 3666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":353,"threads_started":1}
W20260809 15:14:41.075583 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:41.066021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:51383 (local address 127.15.11.190:35905)
0809 15:14:41.066228 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:41.066235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:41.066633 (+ 398us) server_negotiation.cc:407] Connection header received
0809 15:14:41.066769 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:41.066774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:41.066859 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:41.066974 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:41.067751 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.068840 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.070182 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.070597 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.070693 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:41.075321 (+ 4628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:51383: BlockingRecv error: recv got EOF from 127.15.11.129:51383 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 15:14:41 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:41 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:41.311563 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:41.302199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:40323)
0809 15:14:41.302697 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:41.302717 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:41.302833 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:41.303249 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:41.303255 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:41.303794 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:41.303997 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:41.304007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.305177 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.305184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:41.306129 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:41.306142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.306988 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.306995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:41.307217 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:41.307833 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:41.307851 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:41.311340 (+ 3489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":306,"threads_started":1}
W20260809 15:14:41.311847 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:41.302275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:40323 (local address 127.15.11.190:35905)
0809 15:14:41.302511 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:41.302520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:41.302827 (+ 307us) server_negotiation.cc:407] Connection header received
0809 15:14:41.302966 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:41.302972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:41.303062 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:41.303209 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:41.304196 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.305025 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.306303 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.306809 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.306959 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:41.311543 (+ 4584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:40323: BlockingRecv error: recv got EOF from 127.15.11.131:40323 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 15:14:41 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:41 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:41.596400 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:41.587968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:34049)
0809 15:14:41.588512 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:41.588528 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:41.588638 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:41.588927 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:41.588930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:41.589334 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:41.589513 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:41.589521 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.590425 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.590428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:41.591309 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:41.591319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.592007 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.592011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:41.592227 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:41.592806 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:41.592823 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:41.596109 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":301,"threads_started":1}
W20260809 15:14:41.596647 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:41.587989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:34049 (local address 127.15.11.190:35905)
0809 15:14:41.588252 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:41.588261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:41.588608 (+ 347us) server_negotiation.cc:407] Connection header received
0809 15:14:41.588732 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:41.588736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:41.588808 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:41.588885 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:41.589631 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.590299 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.591429 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:41.591861 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:41.591948 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:41.596372 (+ 4424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:34049: BlockingRecv error: recv got EOF from 127.15.11.130:34049 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 15:14:42 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:42 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:42.085456 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:42.076404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:48625)
0809 15:14:42.076897 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:42.076912 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:42.077078 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:42.077402 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:42.077406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:42.077849 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:42.077995 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:42.078004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.079071 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.079081 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:42.080210 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:42.080225 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.080948 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.080954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:42.081126 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:42.081748 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:42.081766 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:42.085092 (+ 3326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":243,"threads_started":1}
W20260809 15:14:42.085522 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:42.076463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:48625 (local address 127.15.11.190:35905)
0809 15:14:42.076686 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:42.076695 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:42.076997 (+ 302us) server_negotiation.cc:407] Connection header received
0809 15:14:42.077207 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:42.077210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:42.077267 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:42.077355 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:42.078144 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.078921 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.080332 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.080785 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.080934 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:42.085292 (+ 4358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:48625: BlockingRecv error: recv got EOF from 127.15.11.129:48625 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 15:14:42 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:42 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:42.321260 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:42.312498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:51663)
0809 15:14:42.313072 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:42.313092 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:42.313250 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:42.313544 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:42.313547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:42.313958 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:42.314122 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:42.314131 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.315085 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.315088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:42.315968 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:42.315978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.316705 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.316709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:42.316827 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:42.317371 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:42.317388 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:42.321035 (+ 3647us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":305,"threads_started":1}
W20260809 15:14:42.321416 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:42.312549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:51663 (local address 127.15.11.190:35905)
0809 15:14:42.312706 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:42.312711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:42.313172 (+ 461us) server_negotiation.cc:407] Connection header received
0809 15:14:42.313350 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:42.313353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:42.313416 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:42.313495 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:42.314242 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.314970 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.316094 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.316560 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.316646 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:42.321218 (+ 4572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:51663: BlockingRecv error: recv got EOF from 127.15.11.131:51663 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 15:14:42 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:42 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:42.606369 25529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:42.597402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:53041)
0809 15:14:42.597833 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:42.597848 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:42.597961 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:42.598257 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:42.598260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:42.598667 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:42.598811 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:42.598819 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.599740 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.599743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:42.600669 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:42.600679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.601627 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.601634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:42.601811 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:42.602479 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:42.602498 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:42.605951 (+ 3453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":230,"threads_started":1}
W20260809 15:14:42.606420 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:42.597503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:53041 (local address 127.15.11.190:35905)
0809 15:14:42.597726 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:42.597732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:42.597922 (+ 190us) server_negotiation.cc:407] Connection header received
0809 15:14:42.598053 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:42.598056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:42.598133 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:42.598215 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:42.598925 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.599622 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.600791 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:42.601498 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:42.601636 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:42.606153 (+ 4517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:53041: BlockingRecv error: recv got EOF from 127.15.11.130:53041 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 15:14:43 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:43 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.129@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:43.100986 25530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:43.086804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.129:42333)
0809 15:14:43.087354 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:43.087383 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:43.087548 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:43.087911 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:43.087918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:43.088379 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:43.088584 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:43.088593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.089665 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.089671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:43.090627 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:43.090639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.091445 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.091450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:43.091632 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:43.097206 (+ 5574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:43.097225 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:43.100642 (+ 3417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":308,"threads_started":1}
W20260809 15:14:43.101084 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:43.086940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:42333 (local address 127.15.11.190:35905)
0809 15:14:43.087169 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:43.087178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:43.087474 (+ 296us) server_negotiation.cc:407] Connection header received
0809 15:14:43.087672 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:43.087678 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:43.087754 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:43.087856 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:43.088748 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.089484 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.090787 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.091275 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.091415 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:43.100843 (+ 9428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.129:42333: BlockingRecv error: recv got EOF from 127.15.11.129:42333 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 15:14:43 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:43 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.131@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:43.331761 25531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:43.322155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.131:41323)
0809 15:14:43.322754 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:43.322779 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:43.322948 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:43.323326 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:43.323330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:43.323750 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:43.323968 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:43.323983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.325033 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.325038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:43.326009 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:43.326021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.326688 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.326694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:43.326875 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:43.327575 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:43.327592 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:43.331530 (+ 3938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":321,"threads_started":1}
W20260809 15:14:43.332024 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:43.322298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:41323 (local address 127.15.11.190:35905)
0809 15:14:43.322669 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:43.322675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:43.322872 (+ 197us) server_negotiation.cc:407] Connection header received
0809 15:14:43.323079 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:43.323088 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:43.323163 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:43.323263 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:43.324158 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.324884 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.326151 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.326544 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.326672 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:43.331737 (+ 5065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.131:41323: BlockingRecv error: recv got EOF from 127.15.11.131:41323 (error 108)
Metrics: {"server-negotiator.queue_time_us":226}
Aug 09 15:14:43 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:14:43 dist-test-slave-8w9v krb5kdc[24362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.11.130@KRBTEST.COM for oryx/127.15.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:14:43.617246 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:43.607556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35905 (local address 127.15.11.130:45845)
0809 15:14:43.608103 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:43.608160 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:43.608329 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:43.608644 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:43.608647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:43.609027 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:43.609210 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:43.609219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.610192 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.610195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:43.611213 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:43.611225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.611904 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.611908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:43.612024 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:43.612643 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:43.612659 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:43.616960 (+ 4301us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.11.190:35905: Server oryx/127.15.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":235,"threads_started":1}
W20260809 15:14:43.617314 25185 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:43.607657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:45845 (local address 127.15.11.190:35905)
0809 15:14:43.607921 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:43.607946 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:43.608263 (+ 317us) server_negotiation.cc:407] Connection header received
0809 15:14:43.608445 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:43.608448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:43.608521 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:43.608600 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:43.609327 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.610065 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.611349 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:43.611767 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:43.611873 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:43.617137 (+ 5264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.11.130:45845: BlockingRecv error: recv got EOF from 127.15.11.130:45845 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 15:14:44.079545 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25019
I20260809 15:14:44.091573 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:44.080704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.15.11.129:38011)
0809 15:14:44.081415 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:44.081422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:44.081450 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:14:44.081511 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:44.081517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:44.081747 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:44.081906 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:44.083270 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:44.084360 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:44.085501 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:44.086016 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:44.086115 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:44.088504 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:44.088513 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:44.088525 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:44.088565 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:44.091048 (+ 2483us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.11.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 15:14:44.091293 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39064: Request ticket server kudu/127.15.11.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":380,"threads_started":1}
W20260809 15:14:44.091686 25534 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39064: Request ticket server kudu/127.15.11.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 15:14:44.091678 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25019: Not authorized: Client connection negotiation failed: client connection to 127.15.11.129:38011: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.11.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 15:14:44.091915 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 25019
I20260809 15:14:44.115505 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25187
I20260809 15:14:44.128183 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:44.116702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.15.11.130:34833)
0809 15:14:44.117415 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:44.117424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:44.117457 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:14:44.117533 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:44.117545 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:44.117942 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:44.118179 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:44.119527 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:44.120532 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:44.121680 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:44.122190 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:44.122285 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:44.124867 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:44.124875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:44.124886 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:44.124920 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:44.127660 (+ 2740us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.11.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 15:14:44.127930 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51352: Request ticket server kudu/127.15.11.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":347,"threads_started":1}
W20260809 15:14:44.128283 25535 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51352: Request ticket server kudu/127.15.11.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 15:14:44.128525 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25187: Not authorized: Client connection negotiation failed: client connection to 127.15.11.130:34833: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.11.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 15:14:44.128712 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 25187
I20260809 15:14:44.150064 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25334
I20260809 15:14:44.163857 25536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:44.151204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59236 (local address 127.15.11.131:40641)
0809 15:14:44.151891 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:44.151899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:44.151925 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:14:44.151992 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:44.152000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:44.152342 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:44.152517 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:44.154408 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:44.155420 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:44.156732 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:44.157318 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:44.157416 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:44.160020 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:44.160029 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:44.160048 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:44.160084 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:44.163193 (+ 3109us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.11.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 15:14:44.163542 (+ 349us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59236: Request ticket server kudu/127.15.11.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":354,"threads_started":1}
W20260809 15:14:44.163859 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25334: Not authorized: Client connection negotiation failed: client connection to 127.15.11.131:40641: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.15.11.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 15:14:44.163960 25536 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59236: Request ticket server kudu/127.15.11.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 15:14:44.164042 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 25334
I20260809 15:14:44.185447 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 24939
I20260809 15:14:44.186646 25011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:44.305660 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 24939
2026-08-09T15:14:44Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21779 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T15:14:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:14:44Z Disabled control of system clock
I20260809 15:14:44.393051 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:33499
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:33499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 15:14:44.728698 25542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:44.728981 25542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:44.729074 25542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:44.736024 25542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:44.736160 25542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:44.736227 25542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:44.736279 25542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:44.736322 25542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:44.745316 25542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:33499
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:33499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:44.746526 25542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:44.748239 25542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:44.761165 25548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:44.761134 25549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:44.761096 25551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:44.762179 25542 server_base.cc:1034] running on GCE node
I20260809 15:14:44.762902 25542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:44.764320 25542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:44.765516 25542 hybrid_clock.cc:648] HybridClock initialized: now 1786288484765462 us; error 52 us; skew 500 ppm
I20260809 15:14:44.765810 25542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:14:44.767962 25542 webserver.cc:460] Webserver started at http://127.15.11.190:36075/ using document root <none> and password file <none>
I20260809 15:14:44.768672 25542 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:44.768759 25542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:44.769033 25542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:44.771844 25542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "7b3da69703ca4932887939bd65a199b1"
format_stamp: "Formatted at 2026-08-09 15:14:44 on dist-test-slave-8w9v"
server_key: "ddd5a4bb173007d23e55d47e57a3e19e"
server_key_iv: "da7d86eb9ebe0603331f68b1bd7e4b5d"
server_key_version: "encryptionkey@0"
I20260809 15:14:44.772626 25542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "7b3da69703ca4932887939bd65a199b1"
format_stamp: "Formatted at 2026-08-09 15:14:44 on dist-test-slave-8w9v"
server_key: "ddd5a4bb173007d23e55d47e57a3e19e"
server_key_iv: "da7d86eb9ebe0603331f68b1bd7e4b5d"
server_key_version: "encryptionkey@0"
I20260809 15:14:44.777742 25542 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:14:44.781060 25557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:44.782290 25542 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:14:44.782477 25542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "7b3da69703ca4932887939bd65a199b1"
format_stamp: "Formatted at 2026-08-09 15:14:44 on dist-test-slave-8w9v"
server_key: "ddd5a4bb173007d23e55d47e57a3e19e"
server_key_iv: "da7d86eb9ebe0603331f68b1bd7e4b5d"
server_key_version: "encryptionkey@0"
I20260809 15:14:44.782692 25542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:44.809340 25542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:44.810137 25542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:44.810429 25542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:44.828682 25542 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:33499
I20260809 15:14:44.828706 25613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:33499 every 8 connection(s)
I20260809 15:14:44.830037 25542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:14:44.833788 25614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:44.834517 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 25542
I20260809 15:14:44.834717 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:14:44.835371 15406 external_mini_cluster.cc:1428] Setting key f7ff8e913d1a2df8147ffe547d89cbb4
I20260809 15:14:44.847687 25614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1: Bootstrap starting.
I20260809 15:14:44.850063 25617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:44.837364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57722 (local address 127.15.11.190:33499)
0809 15:14:44.838433 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:44.838465 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:44.838527 (+ 62us) server_negotiation.cc:407] Connection header received
0809 15:14:44.840218 (+ 1691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:44.840244 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:44.840804 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:44.841260 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:44.841760 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:44.843608 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:44.844939 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:44.846008 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:44.846172 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:44.847289 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:44.847300 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:44.847316 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:44.847969 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:44.848062 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:44.848087 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:44.848217 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:44.848462 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 15:14:44.848783 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":303,"threads_started":1}
I20260809 15:14:44.851373 25614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:44.852499 25614 log.cc:824] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:44.855197 25614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1: No bootstrap required, opened a new log
I20260809 15:14:44.862707 25614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3da69703ca4932887939bd65a199b1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33499 } }
I20260809 15:14:44.863034 25614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:44.863113 25614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b3da69703ca4932887939bd65a199b1, State: Initialized, Role: FOLLOWER
I20260809 15:14:44.863678 25614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3da69703ca4932887939bd65a199b1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33499 } }
I20260809 15:14:44.863902 25614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:14:44.864037 25614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:14:44.864243 25614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:44.866386 25614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3da69703ca4932887939bd65a199b1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33499 } }
I20260809 15:14:44.866868 25614 leader_election.cc:302] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b3da69703ca4932887939bd65a199b1; no voters:
I20260809 15:14:44.867452 25614 leader_election.cc:288] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:14:44.867604 25619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:44.868292 25619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [term 1 LEADER]: Becoming Leader. State: Replica: 7b3da69703ca4932887939bd65a199b1, State: Running, Role: LEADER
I20260809 15:14:44.868929 25619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3da69703ca4932887939bd65a199b1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33499 } }
I20260809 15:14:44.869707 25614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:14:44.871903 25621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b3da69703ca4932887939bd65a199b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3da69703ca4932887939bd65a199b1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33499 } } }
I20260809 15:14:44.872296 25621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:44.871980 25620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b3da69703ca4932887939bd65a199b1. Latest consensus state: current_term: 1 leader_uuid: "7b3da69703ca4932887939bd65a199b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3da69703ca4932887939bd65a199b1" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33499 } } }
I20260809 15:14:44.872419 25620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:44.873991 25625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:14:44.879129 25625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:14:44.887782 25625 catalog_manager.cc:1347] Generated new cluster ID: 5ebe1cb222544683a810d1b731bf9790
I20260809 15:14:44.887887 25625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:14:44.927309 25625 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:14:44.928773 25625 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:14:44.935858 25625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b3da69703ca4932887939bd65a199b1: Generated new TSK 0
I20260809 15:14:44.936708 25625 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:14:44.947259 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33499
--builtin_ntp_servers=127.15.11.148:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 15:14:45.291676 25638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:45.292003 25638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:45.292222 25638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:45.299332 25638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:45.299445 25638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:45.299558 25638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:14:45.308375 25638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:45.309615 25638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:45.311466 25638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:45.325675 25643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:45.325645 25644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:45.325702 25646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:45.326330 25638 server_base.cc:1034] running on GCE node
I20260809 15:14:45.326864 25638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:45.327751 25638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:45.328909 25638 hybrid_clock.cc:648] HybridClock initialized: now 1786288485328843 us; error 47 us; skew 500 ppm
I20260809 15:14:45.329219 25638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:14:45.331549 25638 webserver.cc:460] Webserver started at http://127.15.11.129:39001/ using document root <none> and password file <none>
I20260809 15:14:45.332279 25638 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:45.332373 25638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:45.332643 25638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:45.335427 25638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "ea6270518dc54c13a58fecaba636c2f2"
format_stamp: "Formatted at 2026-08-09 15:14:45 on dist-test-slave-8w9v"
server_key: "d5b53463ffb245e0d124fec1130ad7dd"
server_key_iv: "dc68aeef4084da06d95c348f8f88fd92"
server_key_version: "encryptionkey@0"
I20260809 15:14:45.336226 25638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "ea6270518dc54c13a58fecaba636c2f2"
format_stamp: "Formatted at 2026-08-09 15:14:45 on dist-test-slave-8w9v"
server_key: "d5b53463ffb245e0d124fec1130ad7dd"
server_key_iv: "dc68aeef4084da06d95c348f8f88fd92"
server_key_version: "encryptionkey@0"
I20260809 15:14:45.341048 25638 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:14:45.344400 25652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:45.345685 25638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:14:45.345891 25638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "ea6270518dc54c13a58fecaba636c2f2"
format_stamp: "Formatted at 2026-08-09 15:14:45 on dist-test-slave-8w9v"
server_key: "d5b53463ffb245e0d124fec1130ad7dd"
server_key_iv: "dc68aeef4084da06d95c348f8f88fd92"
server_key_version: "encryptionkey@0"
I20260809 15:14:45.346104 25638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:45.376531 25638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:45.377393 25638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:45.377735 25638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:45.378671 25638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:45.380369 25638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:45.380481 25638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:45.380610 25638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:45.380707 25638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:45.412318 25638 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:41919
I20260809 15:14:45.412385 25784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:41919 every 8 connection(s)
I20260809 15:14:45.413650 25638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:14:45.414645 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 25638
I20260809 15:14:45.414855 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:14:45.415729 15406 external_mini_cluster.cc:1428] Setting key ff9f1e49d5986fcafb0ed4eb3920fdf7
I20260809 15:14:45.419184 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33499
--builtin_ntp_servers=127.15.11.148:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 15:14:45.429209 25787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:45.416714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:43705 (local address 127.15.11.190:33499)
0809 15:14:45.417258 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:45.417262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:45.418754 (+ 1492us) server_negotiation.cc:407] Connection header received
0809 15:14:45.421455 (+ 2701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:45.421459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:45.421521 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:45.421630 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:45.423014 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:45.423704 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:45.426207 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:45.426598 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:45.426690 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:45.428314 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:45.428318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:45.428322 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:45.428339 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:45.428380 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:45.428391 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:45.428461 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:45.428742 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 15:14:45.428817 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":306,"threads_started":1}
I20260809 15:14:45.430510 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:45.417471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:33499 (local address 127.15.11.129:43705)
0809 15:14:45.418584 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:45.418635 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:45.421185 (+ 2550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:45.421894 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:45.421906 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:45.422020 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:45.422845 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:45.422871 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:45.423861 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:45.423869 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:45.426051 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:45.426068 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:45.426820 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:45.426829 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:45.427155 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:45.427909 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:14:45.427914 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:45.427967 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:14:45.428065 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:45.428485 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:45.428517 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:45.428528 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 15:14:45.428713 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 15:14:45.429128 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":350,"threads_started":1}
I20260809 15:14:45.432173 25785 heartbeater.cc:347] Connected to a master server at 127.15.11.190:33499
I20260809 15:14:45.432489 25785 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:45.433341 25785 heartbeater.cc:511] Master 127.15.11.190:33499 requested a full tablet report, sending...
I20260809 15:14:45.435707 25574 ts_manager.cc:194] Registered new tserver with Master: ea6270518dc54c13a58fecaba636c2f2 (127.15.11.129:41919)
I20260809 15:14:45.438004 25574 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.129:43705
W20260809 15:14:45.760917 25789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:45.761183 25789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:45.761262 25789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:45.768229 25789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:45.768312 25789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:45.768415 25789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:14:45.777700 25789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:45.778972 25789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:45.780880 25789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:45.794112 25798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:45.794166 25796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:45.794144 25795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:45.795037 25789 server_base.cc:1034] running on GCE node
I20260809 15:14:45.795693 25789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:45.796711 25789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:45.797935 25789 hybrid_clock.cc:648] HybridClock initialized: now 1786288485797887 us; error 58 us; skew 500 ppm
I20260809 15:14:45.798293 25789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:14:45.800722 25789 webserver.cc:460] Webserver started at http://127.15.11.130:41867/ using document root <none> and password file <none>
I20260809 15:14:45.801399 25789 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:45.801504 25789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:45.801848 25789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:45.804736 25789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "eeb439c23dff4f82977bed7b1ca399c7"
format_stamp: "Formatted at 2026-08-09 15:14:45 on dist-test-slave-8w9v"
server_key: "88b566069d015b25ab526663037ed38a"
server_key_iv: "0b0e03944f9e46b0784c7b51cd774022"
server_key_version: "encryptionkey@0"
I20260809 15:14:45.805522 25789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "eeb439c23dff4f82977bed7b1ca399c7"
format_stamp: "Formatted at 2026-08-09 15:14:45 on dist-test-slave-8w9v"
server_key: "88b566069d015b25ab526663037ed38a"
server_key_iv: "0b0e03944f9e46b0784c7b51cd774022"
server_key_version: "encryptionkey@0"
I20260809 15:14:45.810667 25789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:14:45.814116 25804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:45.815475 25789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:14:45.815642 25789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "eeb439c23dff4f82977bed7b1ca399c7"
format_stamp: "Formatted at 2026-08-09 15:14:45 on dist-test-slave-8w9v"
server_key: "88b566069d015b25ab526663037ed38a"
server_key_iv: "0b0e03944f9e46b0784c7b51cd774022"
server_key_version: "encryptionkey@0"
I20260809 15:14:45.815843 25789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:45.836021 25789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:45.836807 25789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:45.837106 25789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:45.838027 25789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:45.839567 25789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:45.839689 25789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:45.839819 25789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:45.839912 25789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:45.871649 25789 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:44701
I20260809 15:14:45.871737 25930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:44701 every 8 connection(s)
I20260809 15:14:45.873003 25789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:14:45.880591 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 25789
I20260809 15:14:45.880772 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:14:45.881327 15406 external_mini_cluster.cc:1428] Setting key a29f4c2cb72b710f81784c492954f9a0
I20260809 15:14:45.883879 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33499
--builtin_ntp_servers=127.15.11.148:39695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 15:14:45.885654 25787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:45.875769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:33487 (local address 127.15.11.190:33499)
0809 15:14:45.875953 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:45.875957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:45.877327 (+ 1370us) server_negotiation.cc:407] Connection header received
0809 15:14:45.879011 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:45.879016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:45.879084 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:45.879171 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:45.880275 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:45.881384 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:45.882835 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:45.883269 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:45.883408 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:45.884825 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:45.884831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:45.884836 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:45.884866 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:45.884899 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:45.884911 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:45.885009 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:45.885250 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 15:14:45.885345 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:14:45.886619 25933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:45.876273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:33499 (local address 127.15.11.130:33487)
0809 15:14:45.877163 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:45.877216 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:45.878781 (+ 1565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:45.879357 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:45.879373 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:45.879450 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:45.880043 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:45.880064 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:45.881511 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:45.881515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:45.882686 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:45.882698 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:45.883515 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:45.883525 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:45.883737 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:45.884519 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:14:45.884522 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:45.884559 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:14:45.884633 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:45.885009 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:45.885022 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:45.885032 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 15:14:45.885220 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 15:14:45.885603 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":271,"threads_started":1}
I20260809 15:14:45.887974 25931 heartbeater.cc:347] Connected to a master server at 127.15.11.190:33499
I20260809 15:14:45.888293 25931 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:45.889142 25931 heartbeater.cc:511] Master 127.15.11.190:33499 requested a full tablet report, sending...
I20260809 15:14:45.890668 25574 ts_manager.cc:194] Registered new tserver with Master: eeb439c23dff4f82977bed7b1ca399c7 (127.15.11.130:44701)
I20260809 15:14:45.891803 25574 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.130:33487
W20260809 15:14:46.213891 25934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:46.214183 25934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:46.214345 25934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:46.221331 25934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:46.221450 25934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:46.221616 25934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:14:46.230091 25934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:39695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33499
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:46.231418 25934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:46.233162 25934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:46.246337 25940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:46.246358 25943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:46.246457 25941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:46.247102 25934 server_base.cc:1034] running on GCE node
I20260809 15:14:46.247747 25934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:46.248811 25934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:46.250036 25934 hybrid_clock.cc:648] HybridClock initialized: now 1786288486249962 us; error 69 us; skew 500 ppm
I20260809 15:14:46.250428 25934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:14:46.252940 25934 webserver.cc:460] Webserver started at http://127.15.11.131:41255/ using document root <none> and password file <none>
I20260809 15:14:46.253711 25934 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:46.253846 25934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:46.254192 25934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:46.257309 25934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "eb1d3b5d42dd44c1a63d82db16872e80"
format_stamp: "Formatted at 2026-08-09 15:14:46 on dist-test-slave-8w9v"
server_key: "cf698fb1e748fbca93141888ea7b739b"
server_key_iv: "46e4f158500d92fd8f27a68e70260dcc"
server_key_version: "encryptionkey@0"
I20260809 15:14:46.258154 25934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "eb1d3b5d42dd44c1a63d82db16872e80"
format_stamp: "Formatted at 2026-08-09 15:14:46 on dist-test-slave-8w9v"
server_key: "cf698fb1e748fbca93141888ea7b739b"
server_key_iv: "46e4f158500d92fd8f27a68e70260dcc"
server_key_version: "encryptionkey@0"
I20260809 15:14:46.263450 25934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:14:46.266992 25950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:46.268362 25934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:46.268570 25934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "eb1d3b5d42dd44c1a63d82db16872e80"
format_stamp: "Formatted at 2026-08-09 15:14:46 on dist-test-slave-8w9v"
server_key: "cf698fb1e748fbca93141888ea7b739b"
server_key_iv: "46e4f158500d92fd8f27a68e70260dcc"
server_key_version: "encryptionkey@0"
I20260809 15:14:46.268786 25934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:46.303014 25934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:46.303925 25934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:46.304280 25934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:46.305330 25934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:46.306782 25934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:46.306910 25934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:46.307040 25934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:46.307128 25934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:46.340987 25934 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:44741
I20260809 15:14:46.341074 26078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:44741 every 8 connection(s)
I20260809 15:14:46.342334 25934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:14:46.351724 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 25934
I20260809 15:14:46.351881 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:14:46.352519 15406 external_mini_cluster.cc:1428] Setting key e543a59bcd62d1e0b93e32a2c05159b1
I20260809 15:14:46.355576 25787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:46.344962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:46977 (local address 127.15.11.190:33499)
0809 15:14:46.345180 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:46.345186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:46.346690 (+ 1504us) server_negotiation.cc:407] Connection header received
0809 15:14:46.348473 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:46.348478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:46.348548 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:46.348635 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:46.349723 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.350500 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.352403 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.352843 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.352962 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:46.354661 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:46.354666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:46.354669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:46.354686 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:46.354730 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:46.354741 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:46.354878 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:46.355106 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 15:14:46.355199 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:14:46.356647 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:46.345579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:33499 (local address 127.15.11.131:46977)
0809 15:14:46.346521 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:46.346578 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:46.348233 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:46.348835 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:46.348843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:46.348933 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:46.349566 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:46.349584 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.350638 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.350642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:46.352248 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:46.352269 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.353090 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.353095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:46.353339 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:46.354175 (+ 836us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:14:46.354179 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:46.354233 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:14:46.354329 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:46.354867 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:46.354879 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:46.354888 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:14:46.355095 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 15:14:46.355507 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":283,"threads_started":1}
I20260809 15:14:46.357940 26080 heartbeater.cc:347] Connected to a master server at 127.15.11.190:33499
I20260809 15:14:46.358224 26080 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:46.359019 26080 heartbeater.cc:511] Master 127.15.11.190:33499 requested a full tablet report, sending...
I20260809 15:14:46.360400 25574 ts_manager.cc:194] Registered new tserver with Master: eb1d3b5d42dd44c1a63d82db16872e80 (127.15.11.131:44741)
I20260809 15:14:46.361198 25574 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.131:46977
I20260809 15:14:46.367728 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 15:14:46.373840 25787 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:46.372264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.15.11.190:33499)
0809 15:14:46.372540 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:46.372548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:46.372911 (+ 363us) server_negotiation.cc:407] Connection header received
0809 15:14:46.373042 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:46.373046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:46.373131 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:46.373205 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:46.373614 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57730: BlockingRecv error: recv got EOF from 127.0.0.1:57730 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:14:46.374711 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25638
I20260809 15:14:46.381910 26090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:46.375596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52748 (local address 127.15.11.129:41919)
0809 15:14:46.376063 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:46.376070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:46.376094 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:14:46.376198 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:46.376211 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:46.376402 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:46.376521 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:46.376847 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.377824 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.379000 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.379491 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.379581 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:46.380572 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:46.380579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:46.380603 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:46.381175 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:46.381218 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:46.381237 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:46.381335 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:46.381438 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 15:14:46.381565 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":208,"threads_started":1}
I20260809 15:14:46.383066 25780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:46.462054 25785 heartbeater.cc:503] Master 127.15.11.190:33499 was elected leader, sending a full tablet report...
I20260809 15:14:46.463347 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 25638
W20260809 15:14:46.483325 25560 connection.cc:535] server connection from 127.15.11.129:43705 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 15:14:46.483664 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25789
I20260809 15:14:46.491695 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:46.484655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.15.11.130:44701)
0809 15:14:46.485158 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:46.485166 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:46.485191 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:14:46.485249 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:46.485254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:46.485467 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:46.485609 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:46.486023 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.487225 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.488503 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.489096 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.489191 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:46.490248 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:46.490255 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:46.490268 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:46.490856 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:46.490912 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:46.490932 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:46.491015 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:46.491194 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 15:14:46.491340 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":219,"threads_started":1}
I20260809 15:14:46.493036 25925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:46.574221 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 25789
I20260809 15:14:46.595395 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25934
I20260809 15:14:46.603873 26094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:46.596686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.15.11.131:44741)
0809 15:14:46.597346 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:46.597354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:46.597398 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:14:46.597457 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:46.597463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:46.597669 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:46.597796 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:46.598230 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.599266 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.600542 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:46.601133 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:46.601272 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:46.602283 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:46.602292 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:46.602305 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:46.602966 (+ 661us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:46.603014 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:46.603035 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:46.603141 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:46.603326 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:14:46.603507 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":337,"threads_started":1}
I20260809 15:14:46.605188 26074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:46.688776 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 25934
I20260809 15:14:46.707911 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 25542
I20260809 15:14:46.709241 25609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:46.810652 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 25542
2026-08-09T15:14:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2499 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T15:14:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:14:46Z Disabled control of system clock
I20260809 15:14:46.894186 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:43299
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:33395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:43299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 15:14:47.224210 26100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:47.224481 26100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:47.224676 26100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:47.231606 26100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:47.231739 26100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:47.231827 26100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:47.231935 26100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:47.232012 26100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:47.241086 26100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:33395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:43299
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:43299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:47.242308 26100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:47.244105 26100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:47.256090 26105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:47.256011 26106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:47.256059 26109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:47.256821 26100 server_base.cc:1034] running on GCE node
I20260809 15:14:47.257670 26100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:47.259095 26100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:47.260358 26100 hybrid_clock.cc:648] HybridClock initialized: now 1786288487260308 us; error 60 us; skew 500 ppm
I20260809 15:14:47.260732 26100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:14:47.263093 26100 webserver.cc:460] Webserver started at http://127.15.11.190:39721/ using document root <none> and password file <none>
I20260809 15:14:47.263808 26100 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:47.263953 26100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:47.264356 26100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:47.269053 26100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f"
format_stamp: "Formatted at 2026-08-09 15:14:47 on dist-test-slave-8w9v"
server_key: "0cb4270d24247270e36dd8e14db752ff"
server_key_iv: "89eb480ce1cb52b30a3c15946067c0c0"
server_key_version: "encryptionkey@0"
I20260809 15:14:47.269856 26100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f"
format_stamp: "Formatted at 2026-08-09 15:14:47 on dist-test-slave-8w9v"
server_key: "0cb4270d24247270e36dd8e14db752ff"
server_key_iv: "89eb480ce1cb52b30a3c15946067c0c0"
server_key_version: "encryptionkey@0"
I20260809 15:14:47.274858 26100 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260809 15:14:47.278329 26115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:47.279542 26100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:14:47.279722 26100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f"
format_stamp: "Formatted at 2026-08-09 15:14:47 on dist-test-slave-8w9v"
server_key: "0cb4270d24247270e36dd8e14db752ff"
server_key_iv: "89eb480ce1cb52b30a3c15946067c0c0"
server_key_version: "encryptionkey@0"
I20260809 15:14:47.279945 26100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:47.285465 26100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:47.286185 26100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:47.286486 26100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:47.304893 26100 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:43299
I20260809 15:14:47.304934 26171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:43299 every 8 connection(s)
I20260809 15:14:47.306291 26100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:14:47.307814 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 26100
I20260809 15:14:47.308048 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:14:47.309044 15406 external_mini_cluster.cc:1428] Setting key 269e0d270e0e585ac947f2cb679d78d5
I20260809 15:14:47.311267 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:47.317214 26175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:47.311159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59710 (local address 127.15.11.190:43299)
0809 15:14:47.312276 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:47.312285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:47.312327 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:14:47.313743 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:47.313762 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:47.314164 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:47.314480 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:47.315651 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:47.315656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:47.315668 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:47.315680 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:47.315749 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:47.315763 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:47.315930 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:47.316032 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 15:14:47.316349 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":418,"threads_started":1}
I20260809 15:14:47.322626 26172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f: Bootstrap starting.
I20260809 15:14:47.325528 26172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:47.326450 26172 log.cc:824] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:47.329275 26172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f: No bootstrap required, opened a new log
I20260809 15:14:47.336469 26172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43299 } }
I20260809 15:14:47.336807 26172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:47.336930 26172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b396653634c43f1b1bc2cd0c7ab4b9f, State: Initialized, Role: FOLLOWER
I20260809 15:14:47.337469 26172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43299 } }
I20260809 15:14:47.337697 26172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:14:47.337831 26172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:14:47.337994 26172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:47.340292 26172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43299 } }
I20260809 15:14:47.340749 26172 leader_election.cc:302] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b396653634c43f1b1bc2cd0c7ab4b9f; no voters:
I20260809 15:14:47.341306 26172 leader_election.cc:288] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:14:47.341451 26177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:47.342170 26177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [term 1 LEADER]: Becoming Leader. State: Replica: 8b396653634c43f1b1bc2cd0c7ab4b9f, State: Running, Role: LEADER
I20260809 15:14:47.342689 26177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43299 } }
I20260809 15:14:47.343284 26172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:14:47.345474 26179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b396653634c43f1b1bc2cd0c7ab4b9f. Latest consensus state: current_term: 1 leader_uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43299 } } }
I20260809 15:14:47.345929 26179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:47.346729 26178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b396653634c43f1b1bc2cd0c7ab4b9f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43299 } } }
I20260809 15:14:47.347204 26178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:47.348374 26186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:14:47.352936 26186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:14:47.361640 26186 catalog_manager.cc:1347] Generated new cluster ID: 8bc35dd1a45540f58687705676a94367
I20260809 15:14:47.361778 26186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:14:47.387177 26186 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:14:47.387599 26186 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:14:47.394085 26186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b396653634c43f1b1bc2cd0c7ab4b9f: Generated new TSK 0
I20260809 15:14:47.394817 26186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:14:47.400707 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43299
--builtin_ntp_servers=127.15.11.148:33395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 15:14:47.739513 26197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:47.739768 26197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:47.739916 26197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:47.747427 26197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:47.747543 26197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:47.747699 26197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:14:47.758478 26197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:33395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:47.759760 26197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:47.761576 26197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:47.774706 26204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:47.774679 26203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:47.774739 26206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:47.775502 26197 server_base.cc:1034] running on GCE node
I20260809 15:14:47.776098 26197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:47.777046 26197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:47.778292 26197 hybrid_clock.cc:648] HybridClock initialized: now 1786288487778234 us; error 58 us; skew 500 ppm
I20260809 15:14:47.778615 26197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:14:47.780882 26197 webserver.cc:460] Webserver started at http://127.15.11.129:46547/ using document root <none> and password file <none>
I20260809 15:14:47.781579 26197 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:47.781715 26197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:47.782045 26197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:47.785006 26197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "b78ee9c745484272b07201b9531d25c7"
format_stamp: "Formatted at 2026-08-09 15:14:47 on dist-test-slave-8w9v"
server_key: "f22ec94c8d22d8e94884f86e3962f067"
server_key_iv: "b27f2d172c8ed82cd02bc9ecca9212d5"
server_key_version: "encryptionkey@0"
I20260809 15:14:47.785821 26197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "b78ee9c745484272b07201b9531d25c7"
format_stamp: "Formatted at 2026-08-09 15:14:47 on dist-test-slave-8w9v"
server_key: "f22ec94c8d22d8e94884f86e3962f067"
server_key_iv: "b27f2d172c8ed82cd02bc9ecca9212d5"
server_key_version: "encryptionkey@0"
I20260809 15:14:47.790795 26197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 15:14:47.794113 26213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:47.795341 26197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:14:47.795522 26197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "b78ee9c745484272b07201b9531d25c7"
format_stamp: "Formatted at 2026-08-09 15:14:47 on dist-test-slave-8w9v"
server_key: "f22ec94c8d22d8e94884f86e3962f067"
server_key_iv: "b27f2d172c8ed82cd02bc9ecca9212d5"
server_key_version: "encryptionkey@0"
I20260809 15:14:47.795743 26197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:47.801213 26197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:47.801993 26197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:47.802276 26197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:47.803169 26197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:47.804608 26197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:47.804730 26197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:47.804874 26197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:47.804968 26197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:47.838001 26197 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:40021
I20260809 15:14:47.838099 26339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:40021 every 8 connection(s)
I20260809 15:14:47.839344 26197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:14:47.847793 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:47.842002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:56451 (local address 127.15.11.190:43299)
0809 15:14:47.842432 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:47.842439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:47.843698 (+ 1259us) server_negotiation.cc:407] Connection header received
0809 15:14:47.845470 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:47.845474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:47.845527 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:47.845622 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:47.846986 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:47.846990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:47.846993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:47.847008 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:47.847046 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:47.847057 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:47.847148 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:47.847375 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 15:14:47.847426 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":198,"threads_started":1}
I20260809 15:14:47.848590 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 26197
I20260809 15:14:47.848807 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:14:47.848789 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:47.842638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43299 (local address 127.15.11.129:56451)
0809 15:14:47.843540 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:47.843590 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:47.845234 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:47.845820 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:47.845828 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:47.845926 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:47.846708 (+ 782us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:14:47.846710 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:47.846748 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:14:47.846833 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:47.847173 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:47.847186 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:47.847194 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:14:47.847426 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 15:14:47.847664 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":284,"threads_started":1}
I20260809 15:14:47.849642 15406 external_mini_cluster.cc:1428] Setting key d804e366a708f2c362aed2441348da4d
I20260809 15:14:47.850373 26341 heartbeater.cc:347] Connected to a master server at 127.15.11.190:43299
I20260809 15:14:47.850697 26341 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:47.851289 26341 heartbeater.cc:511] Master 127.15.11.190:43299 requested a full tablet report, sending...
I20260809 15:14:47.852766 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43299
--builtin_ntp_servers=127.15.11.148:33395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 15:14:47.853443 26133 ts_manager.cc:194] Registered new tserver with Master: b78ee9c745484272b07201b9531d25c7 (127.15.11.129:40021)
W20260809 15:14:48.179694 26345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:48.179955 26345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:48.180142 26345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:48.187408 26345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:48.187531 26345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:48.187701 26345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:14:48.196879 26345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:33395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:48.198168 26345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:48.200032 26345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:48.213389 26350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:48.213389 26354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:48.213591 26351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:48.214145 26345 server_base.cc:1034] running on GCE node
I20260809 15:14:48.214783 26345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:48.215698 26345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:48.216928 26345 hybrid_clock.cc:648] HybridClock initialized: now 1786288488216881 us; error 57 us; skew 500 ppm
I20260809 15:14:48.217248 26345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:14:48.223485 26345 webserver.cc:460] Webserver started at http://127.15.11.130:40529/ using document root <none> and password file <none>
I20260809 15:14:48.224277 26345 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:48.224414 26345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:48.224745 26345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:48.227485 26345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "a7c681257e6340108b554facd947b4d8"
format_stamp: "Formatted at 2026-08-09 15:14:48 on dist-test-slave-8w9v"
server_key: "b2aebafe0cc9d880c8087de6407b7836"
server_key_iv: "43d3336270f58b921e85bd9f83118697"
server_key_version: "encryptionkey@0"
I20260809 15:14:48.228499 26345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "a7c681257e6340108b554facd947b4d8"
format_stamp: "Formatted at 2026-08-09 15:14:48 on dist-test-slave-8w9v"
server_key: "b2aebafe0cc9d880c8087de6407b7836"
server_key_iv: "43d3336270f58b921e85bd9f83118697"
server_key_version: "encryptionkey@0"
I20260809 15:14:48.233281 26345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:14:48.236565 26363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:48.237818 26345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:14:48.238003 26345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "a7c681257e6340108b554facd947b4d8"
format_stamp: "Formatted at 2026-08-09 15:14:48 on dist-test-slave-8w9v"
server_key: "b2aebafe0cc9d880c8087de6407b7836"
server_key_iv: "43d3336270f58b921e85bd9f83118697"
server_key_version: "encryptionkey@0"
I20260809 15:14:48.238245 26345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:48.243695 26345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:48.244573 26345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:48.244840 26345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:48.245704 26345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:48.247136 26345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:48.247221 26345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:48.247299 26345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:48.247352 26345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:48.280336 26345 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:39445
I20260809 15:14:48.280465 26490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:39445 every 8 connection(s)
I20260809 15:14:48.281678 26345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:14:48.289274 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 26345
I20260809 15:14:48.289480 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:14:48.290020 15406 external_mini_cluster.cc:1428] Setting key 988490d426e3f2aae22257cc6a51521c
I20260809 15:14:48.290597 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:48.284390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:54597 (local address 127.15.11.190:43299)
0809 15:14:48.284595 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:48.284601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:48.286045 (+ 1444us) server_negotiation.cc:407] Connection header received
0809 15:14:48.287796 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:48.287800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:48.287864 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:48.287960 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:48.289771 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:48.289778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:48.289785 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:48.289808 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:48.289841 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:48.289854 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:48.289997 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:48.290216 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 15:14:48.290289 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:14:48.291589 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:48.284966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43299 (local address 127.15.11.130:54597)
0809 15:14:48.285891 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:48.285930 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:48.287546 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:48.288238 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:48.288251 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:48.288435 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:48.289268 (+ 833us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:14:48.289270 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:48.289312 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:14:48.289484 (+ 172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:48.289969 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:48.289982 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:48.289990 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:14:48.290315 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 15:14:48.290583 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":278,"threads_started":1}
I20260809 15:14:48.292908 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43299
--builtin_ntp_servers=127.15.11.148:33395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 15:14:48.293228 26491 heartbeater.cc:347] Connected to a master server at 127.15.11.190:43299
I20260809 15:14:48.293581 26491 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:48.294195 26491 heartbeater.cc:511] Master 127.15.11.190:43299 requested a full tablet report, sending...
I20260809 15:14:48.295678 26133 ts_manager.cc:194] Registered new tserver with Master: a7c681257e6340108b554facd947b4d8 (127.15.11.130:39445)
W20260809 15:14:48.635048 26494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:48.635308 26494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:48.635397 26494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:48.642338 26494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:48.642427 26494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:48.642524 26494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:14:48.651366 26494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:33395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:48.652575 26494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:48.654256 26494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:48.667112 26502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:48.667066 26505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:48.667158 26499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:48.667979 26494 server_base.cc:1034] running on GCE node
I20260809 15:14:48.668615 26494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:48.669785 26494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:48.671031 26494 hybrid_clock.cc:648] HybridClock initialized: now 1786288488670959 us; error 54 us; skew 500 ppm
I20260809 15:14:48.671414 26494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:14:48.673971 26494 webserver.cc:460] Webserver started at http://127.15.11.131:36133/ using document root <none> and password file <none>
I20260809 15:14:48.674667 26494 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:48.674784 26494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:48.675107 26494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:48.678071 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "1111aee61bc64e289d7c0e5eb65d3c34"
format_stamp: "Formatted at 2026-08-09 15:14:48 on dist-test-slave-8w9v"
server_key: "cf7118127ed8aa961e94074e3c4cc7ff"
server_key_iv: "6ada71fa7b9f050ad7f52b09d2b01e03"
server_key_version: "encryptionkey@0"
I20260809 15:14:48.678874 26494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "1111aee61bc64e289d7c0e5eb65d3c34"
format_stamp: "Formatted at 2026-08-09 15:14:48 on dist-test-slave-8w9v"
server_key: "cf7118127ed8aa961e94074e3c4cc7ff"
server_key_iv: "6ada71fa7b9f050ad7f52b09d2b01e03"
server_key_version: "encryptionkey@0"
I20260809 15:14:48.684432 26494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 15:14:48.687819 26511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:48.689158 26494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:48.689355 26494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "1111aee61bc64e289d7c0e5eb65d3c34"
format_stamp: "Formatted at 2026-08-09 15:14:48 on dist-test-slave-8w9v"
server_key: "cf7118127ed8aa961e94074e3c4cc7ff"
server_key_iv: "6ada71fa7b9f050ad7f52b09d2b01e03"
server_key_version: "encryptionkey@0"
I20260809 15:14:48.689592 26494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:48.695263 26494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:48.696050 26494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:48.696334 26494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:48.697235 26494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:48.698675 26494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:48.698797 26494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:48.698935 26494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:48.699038 26494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:48.732748 26494 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:34403
I20260809 15:14:48.732841 26650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:34403 every 8 connection(s)
I20260809 15:14:48.734087 26494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:14:48.742599 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 26494
I20260809 15:14:48.742642 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:48.736917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:45777 (local address 127.15.11.190:43299)
0809 15:14:48.737116 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:48.737120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:48.738575 (+ 1455us) server_negotiation.cc:407] Connection header received
0809 15:14:48.740319 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:48.740324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:48.740400 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:48.740510 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:48.741874 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:48.741879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:48.741884 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:48.741908 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:48.741940 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:48.741951 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:48.742084 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:48.742269 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:14:48.742340 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:14:48.742827 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:14:48.743346 15406 external_mini_cluster.cc:1428] Setting key e55b323854f280bc34be2d641666edd5
I20260809 15:14:48.743755 26653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:48.737476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43299 (local address 127.15.11.131:45777)
0809 15:14:48.738418 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:48.738454 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:48.740010 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:48.740735 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:48.740743 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:48.740828 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:48.741528 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:14:48.741531 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:48.741575 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:14:48.741688 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:48.742047 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:48.742059 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:48.742069 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 15:14:48.742350 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 15:14:48.742603 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":294,"threads_started":1}
I20260809 15:14:48.745380 26651 heartbeater.cc:347] Connected to a master server at 127.15.11.190:43299
I20260809 15:14:48.745774 26651 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:48.746517 26651 heartbeater.cc:511] Master 127.15.11.190:43299 requested a full tablet report, sending...
I20260809 15:14:48.747979 26132 ts_manager.cc:194] Registered new tserver with Master: 1111aee61bc64e289d7c0e5eb65d3c34 (127.15.11.131:34403)
I20260809 15:14:48.758895 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 15:14:48.765153 26343 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:14:48.763405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.15.11.190:43299)
0809 15:14:48.763582 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:48.763598 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:48.764143 (+ 545us) server_negotiation.cc:407] Connection header received
0809 15:14:48.764340 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:48.764346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:48.764409 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:48.764509 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:48.764884 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59722: BlockingRecv error: recv got EOF from 127.0.0.1:59722 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:14:48.765719 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 26197
I20260809 15:14:48.769361 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:48.766739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45768 (local address 127.15.11.129:40021)
0809 15:14:48.767262 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:48.767278 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:48.767303 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:14:48.767365 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:48.767371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:48.767566 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:48.767685 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:48.768666 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:48.768672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:48.768683 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:48.768703 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:48.768744 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:48.768758 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:48.768878 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:48.768952 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 15:14:48.769062 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":259,"threads_started":1}
I20260809 15:14:48.770597 26333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:48.849041 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 26197
I20260809 15:14:48.870404 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 26345
I20260809 15:14:48.873993 26665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:48.871336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.15.11.130:39445)
0809 15:14:48.871933 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:48.871939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:48.871966 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:14:48.872030 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:48.872036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:48.872317 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:48.872449 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:48.873286 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:48.873292 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:48.873304 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:48.873322 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:48.873366 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:48.873381 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:48.873485 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:48.873583 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 15:14:48.873701 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":311,"threads_started":1}
I20260809 15:14:48.875268 26482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:48.953557 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 26345
I20260809 15:14:48.973554 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 26494
I20260809 15:14:48.977392 26667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:48.974588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.15.11.131:34403)
0809 15:14:48.975183 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:48.975190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:48.975220 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:14:48.975281 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:48.975287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:48.975525 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:48.975646 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:48.976719 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:48.976725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:48.976737 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:14:48.976753 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:48.976792 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:14:48.976805 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:48.976964 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:48.977018 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 15:14:48.977098 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":291,"threads_started":1}
I20260809 15:14:48.978674 26642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:49.060154 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 26494
I20260809 15:14:49.080348 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 26100
I20260809 15:14:49.081588 26166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:49.182046 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 26100
2026-08-09T15:14:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2371 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:14:49 dist-test-slave-8w9v krb5kdc[26672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:14:49 dist-test-slave-8w9v krb5kdc[26672](info): set up 2 sockets
Aug 09 15:14:49 dist-test-slave-8w9v krb5kdc[26672](info): commencing operation
krb5kdc: starting...
W20260809 15:14:50.345345 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.005s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:14:50 dist-test-slave-8w9v krb5kdc[26672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:14:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:14:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:14:50.596731 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:36091
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:36091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:50.923884 26687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:50.924219 26687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:50.924376 26687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:50.931180 26687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:14:50.931315 26687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:50.931424 26687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:50.931519 26687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:14:50.931583 26687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:14:50.940805 26687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:36091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:36091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:50.942073 26687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:50.943377 26687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:50.955068 26692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:50.954978 26696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:50.955025 26693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:50.956235 26687 server_base.cc:1034] running on GCE node
I20260809 15:14:50.956952 26687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:50.958297 26687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:50.959553 26687 hybrid_clock.cc:648] HybridClock initialized: now 1786288490959510 us; error 53 us; skew 500 ppm
I20260809 15:14:50.959884 26687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:50 dist-test-slave-8w9v krb5kdc[26672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288490, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:50.962977 26687 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:14:50.964371 26687 webserver.cc:460] Webserver started at http://127.15.11.190:45865/ using document root <none> and password file <none>
I20260809 15:14:50.965027 26687 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:50.965174 26687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:50.965485 26687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:50.968540 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "11a289e9cabf499ead68b7f15e8c0606"
format_stamp: "Formatted at 2026-08-09 15:14:50 on dist-test-slave-8w9v"
server_key: "b9607034804a62841b09538107133908"
server_key_iv: "81c41661eb19bc2832eb873b66cdcee9"
server_key_version: "encryptionkey@0"
I20260809 15:14:50.969295 26687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "11a289e9cabf499ead68b7f15e8c0606"
format_stamp: "Formatted at 2026-08-09 15:14:50 on dist-test-slave-8w9v"
server_key: "b9607034804a62841b09538107133908"
server_key_iv: "81c41661eb19bc2832eb873b66cdcee9"
server_key_version: "encryptionkey@0"
I20260809 15:14:50.974229 26687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 15:14:50.977523 26703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:50.978685 26687 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:14:50.978859 26687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "11a289e9cabf499ead68b7f15e8c0606"
format_stamp: "Formatted at 2026-08-09 15:14:50 on dist-test-slave-8w9v"
server_key: "b9607034804a62841b09538107133908"
server_key_iv: "81c41661eb19bc2832eb873b66cdcee9"
server_key_version: "encryptionkey@0"
I20260809 15:14:50.979056 26687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:51.004791 26687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:51.008435 26687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:51.008718 26687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:51.026449 26687 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:36091
I20260809 15:14:51.026479 26759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:36091 every 8 connection(s)
I20260809 15:14:51.027802 26687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:14:51.030582 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 26687
I20260809 15:14:51.030829 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:14:51.031486 15406 external_mini_cluster.cc:1428] Setting key 934a5a1eaa6048ae312379ab2d391322
I20260809 15:14:51.031528 26760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:51.045964 26760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606: Bootstrap starting.
Aug 09 15:14:51 dist-test-slave-8w9v krb5kdc[26672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:51.049916 26760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:51.051314 26760 log.cc:824] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:51.054826 26760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606: No bootstrap required, opened a new log
I20260809 15:14:51.055823 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:51.033431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36290 (local address 127.15.11.190:36091)
0809 15:14:51.034218 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:51.034229 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:51.034272 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:14:51.036082 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:51.036103 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:51.036548 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:51.036876 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:51.038738 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:51.039995 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:51.041756 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:51.042432 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:51.042569 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:51.047185 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:51.047195 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:51.047212 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:51.047268 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:51.051314 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:51.052105 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:51.052112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:51.052117 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:51.052339 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:51.052776 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:51.052780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:51.052783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:51.053459 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:51.053801 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:51.054084 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 15:14:51.054514 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":355,"threads_started":1}
I20260809 15:14:51.062919 26760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a289e9cabf499ead68b7f15e8c0606" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36091 } }
I20260809 15:14:51.063331 26760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:51.063452 26760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11a289e9cabf499ead68b7f15e8c0606, State: Initialized, Role: FOLLOWER
I20260809 15:14:51.064054 26760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a289e9cabf499ead68b7f15e8c0606" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36091 } }
I20260809 15:14:51.064356 26760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:14:51.064525 26760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:14:51.064703 26760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:51.066741 26760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a289e9cabf499ead68b7f15e8c0606" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36091 } }
I20260809 15:14:51.067241 26760 leader_election.cc:302] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11a289e9cabf499ead68b7f15e8c0606; no voters:
I20260809 15:14:51.067785 26760 leader_election.cc:288] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:14:51.067890 26766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:51.069140 26766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [term 1 LEADER]: Becoming Leader. State: Replica: 11a289e9cabf499ead68b7f15e8c0606, State: Running, Role: LEADER
I20260809 15:14:51.069667 26766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a289e9cabf499ead68b7f15e8c0606" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36091 } }
I20260809 15:14:51.069967 26760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:14:51.072687 26767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11a289e9cabf499ead68b7f15e8c0606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a289e9cabf499ead68b7f15e8c0606" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36091 } } }
I20260809 15:14:51.073099 26767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:51.073127 26769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11a289e9cabf499ead68b7f15e8c0606. Latest consensus state: current_term: 1 leader_uuid: "11a289e9cabf499ead68b7f15e8c0606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11a289e9cabf499ead68b7f15e8c0606" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 36091 } } }
I20260809 15:14:51.073460 26769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606 [sys.catalog]: This master's current role is: LEADER
I20260809 15:14:51.074504 26776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:14:51.079371 26776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:14:51.087788 26776 catalog_manager.cc:1347] Generated new cluster ID: d6b7e8dc1f4947b4b70435e36e147643
I20260809 15:14:51.087916 26776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:14:51.099107 26776 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:14:51.100347 26776 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:14:51.107805 26776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606: Generated new TSK 0
I20260809 15:14:51.108628 26776 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:14:51.184230 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:36091
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:51.523008 26790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:51.523317 26790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:51.523499 26790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:51.530689 26790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:51.530843 26790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:51.531030 26790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:14:51.540472 26790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:36091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:51.541762 26790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:51.543078 26790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:51.556556 26796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:51.556515 26800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:51.556754 26795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:51.557988 26790 server_base.cc:1034] running on GCE node
I20260809 15:14:51.558507 26790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:51.559410 26790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:51.560603 26790 hybrid_clock.cc:648] HybridClock initialized: now 1786288491560548 us; error 54 us; skew 500 ppm
I20260809 15:14:51.560925 26790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:51 dist-test-slave-8w9v krb5kdc[26672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288491, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:51.563997 26790 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:14:51.565341 26790 webserver.cc:460] Webserver started at http://127.15.11.129:34897/ using document root <none> and password file <none>
I20260809 15:14:51.565970 26790 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:51.566069 26790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:51.566304 26790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:51.569097 26790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "27128422655744cd9cf2d31705005420"
format_stamp: "Formatted at 2026-08-09 15:14:51 on dist-test-slave-8w9v"
server_key: "aef02cc052a656cd6b99c6cf4d16a697"
server_key_iv: "785c44932aaaa6d64876266cde2cfb76"
server_key_version: "encryptionkey@0"
I20260809 15:14:51.569903 26790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "27128422655744cd9cf2d31705005420"
format_stamp: "Formatted at 2026-08-09 15:14:51 on dist-test-slave-8w9v"
server_key: "aef02cc052a656cd6b99c6cf4d16a697"
server_key_iv: "785c44932aaaa6d64876266cde2cfb76"
server_key_version: "encryptionkey@0"
I20260809 15:14:51.575237 26790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 15:14:51.578604 26807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:51.579847 26790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:14:51.580050 26790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "27128422655744cd9cf2d31705005420"
format_stamp: "Formatted at 2026-08-09 15:14:51 on dist-test-slave-8w9v"
server_key: "aef02cc052a656cd6b99c6cf4d16a697"
server_key_iv: "785c44932aaaa6d64876266cde2cfb76"
server_key_version: "encryptionkey@0"
I20260809 15:14:51.580283 26790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:51.606076 26790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:51.610271 26790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:51.610605 26790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:51.611546 26790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:51.612910 26790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:51.613008 26790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:51.613149 26790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:51.613220 26790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:51.646121 26790 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:41907
I20260809 15:14:51.646162 26935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:41907 every 8 connection(s)
I20260809 15:14:51.647384 26790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:14:51.651090 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 26790
I20260809 15:14:51.651342 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:14:51.652236 15406 external_mini_cluster.cc:1428] Setting key 84da06ea788c7ce741b3ece5673c8cbd
Aug 09 15:14:51 dist-test-slave-8w9v krb5kdc[26672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288491, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:51.669967 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:51.649943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:36671 (local address 127.15.11.190:36091)
0809 15:14:51.650432 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:51.650437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:51.651428 (+ 991us) server_negotiation.cc:407] Connection header received
0809 15:14:51.653707 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:51.653713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:51.653804 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:51.653936 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:51.655995 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:51.657206 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:51.659628 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:51.660014 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:51.660113 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:51.664336 (+ 4223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:51.664341 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:51.664344 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:51.664376 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:51.666903 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:51.667618 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:51.667625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:51.667628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:51.667756 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:51.668227 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:51.668235 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:51.668239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:51.668515 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:51.668670 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:51.669362 (+ 692us) server_negotiation.cc:299] Negotiation successful
0809 15:14:51.669587 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":262,"threads_started":1}
I20260809 15:14:51.670876 26939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:51.650344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:36091 (local address 127.15.11.129:36671)
0809 15:14:51.651211 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:51.651274 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:51.653441 (+ 2167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:51.654176 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:51.654187 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:51.654952 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:51.655764 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:51.655808 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:51.657371 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:51.657379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:51.659473 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:51.659497 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:51.660272 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:51.660281 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:51.660602 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:51.661325 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:51.661359 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:51.664074 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:51.667053 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:51.667062 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:51.667076 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:51.667497 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:51.667871 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:51.667875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:51.667879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:51.668076 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:51.668645 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:51.668653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:51.668973 (+ 320us) client_negotiation.cc:769] Sending connection context
0809 15:14:51.669234 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 15:14:51.669758 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":309,"threads_started":1}
I20260809 15:14:51.672565 26936 heartbeater.cc:347] Connected to a master server at 127.15.11.190:36091
I20260809 15:14:51.672909 26936 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:51.673658 26936 heartbeater.cc:511] Master 127.15.11.190:36091 requested a full tablet report, sending...
I20260809 15:14:51.676102 26721 ts_manager.cc:194] Registered new tserver with Master: 27128422655744cd9cf2d31705005420 (127.15.11.129:41907)
I20260809 15:14:51.678423 26721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:36671
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:14:51.715551 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:36091
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:52.059233 26943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:52.059543 26943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:52.059707 26943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:52.066797 26943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:52.066941 26943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:52.067119 26943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:14:52.076874 26943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:36091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:52.078198 26943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:52.079511 26943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:52.092923 26949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:52.093003 26950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:52.093029 26953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:52.093701 26943 server_base.cc:1034] running on GCE node
I20260809 15:14:52.094322 26943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:52.095232 26943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:52.096463 26943 hybrid_clock.cc:648] HybridClock initialized: now 1786288492096398 us; error 49 us; skew 500 ppm
I20260809 15:14:52.096769 26943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:52 dist-test-slave-8w9v krb5kdc[26672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288492, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:52.099835 26943 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:14:52.101332 26943 webserver.cc:460] Webserver started at http://127.15.11.130:32825/ using document root <none> and password file <none>
I20260809 15:14:52.102046 26943 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:52.102188 26943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:52.102536 26943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:52.105484 26943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "5f4c0804512e4b03b9d87361101fc4e1"
format_stamp: "Formatted at 2026-08-09 15:14:52 on dist-test-slave-8w9v"
server_key: "c942033514af7300648cb245a1f9cefa"
server_key_iv: "57861198a76d252c3306341b3dc8d4c9"
server_key_version: "encryptionkey@0"
I20260809 15:14:52.106256 26943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "5f4c0804512e4b03b9d87361101fc4e1"
format_stamp: "Formatted at 2026-08-09 15:14:52 on dist-test-slave-8w9v"
server_key: "c942033514af7300648cb245a1f9cefa"
server_key_iv: "57861198a76d252c3306341b3dc8d4c9"
server_key_version: "encryptionkey@0"
I20260809 15:14:52.111268 26943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 15:14:52.114533 26962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:52.115770 26943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:14:52.115943 26943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "5f4c0804512e4b03b9d87361101fc4e1"
format_stamp: "Formatted at 2026-08-09 15:14:52 on dist-test-slave-8w9v"
server_key: "c942033514af7300648cb245a1f9cefa"
server_key_iv: "57861198a76d252c3306341b3dc8d4c9"
server_key_version: "encryptionkey@0"
I20260809 15:14:52.116189 26943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:52.135251 26943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:52.138939 26943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:52.139249 26943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:52.140228 26943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:52.141584 26943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:52.141685 26943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:52.141839 26943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:52.141913 26943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:52.181517 26943 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:38211
I20260809 15:14:52.181568 27103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:38211 every 8 connection(s)
I20260809 15:14:52.183321 26943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:14:52.190882 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 26943
I20260809 15:14:52.191128 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:14:52.191762 15406 external_mini_cluster.cc:1428] Setting key e368291f3e85592a4ea6986f8bd3e4d0
Aug 09 15:14:52 dist-test-slave-8w9v krb5kdc[26672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288492, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:52.210582 27107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.186293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:36623 (local address 127.15.11.190:36091)
0809 15:14:52.186800 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:52.186805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:52.188160 (+ 1355us) server_negotiation.cc:407] Connection header received
0809 15:14:52.191372 (+ 3212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:52.191379 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:52.191471 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:52.191594 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:52.194309 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.195557 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.198140 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.198954 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.199091 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.203666 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:52.203675 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:52.203682 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:52.203730 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:52.207087 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:52.207863 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:52.207868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:52.207872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:52.208013 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:52.208630 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:52.208634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:52.208637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:52.209090 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:52.209249 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:52.209943 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 15:14:52.210164 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":279,"threads_started":1}
I20260809 15:14:52.211664 27108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.187029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:36091 (local address 127.15.11.130:36623)
0809 15:14:52.187963 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:52.188019 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:52.190901 (+ 2882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:52.192082 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:52.192094 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:52.193349 (+ 1255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:52.194106 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.194132 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.195764 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.195773 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.197993 (+ 2220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.198011 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.199229 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.199238 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.199569 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.200648 (+ 1079us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:52.200677 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:52.203377 (+ 2700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:52.207247 (+ 3870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:52.207255 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:52.207272 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:52.207730 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:52.208151 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:52.208159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:52.208165 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:52.208480 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:52.209223 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:52.209231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:52.209587 (+ 356us) client_negotiation.cc:769] Sending connection context
0809 15:14:52.209856 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 15:14:52.210411 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":370,"threads_started":1}
I20260809 15:14:52.213336 27105 heartbeater.cc:347] Connected to a master server at 127.15.11.190:36091
I20260809 15:14:52.213719 27105 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:52.214771 27105 heartbeater.cc:511] Master 127.15.11.190:36091 requested a full tablet report, sending...
I20260809 15:14:52.216706 26721 ts_manager.cc:194] Registered new tserver with Master: 5f4c0804512e4b03b9d87361101fc4e1 (127.15.11.130:38211)
I20260809 15:14:52.217919 26721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:36623
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:14:52.261711 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:36091
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:14:52.599707 27114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:14:52.600013 27114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:14:52.600203 27114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:14:52.607407 27114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:14:52.607559 27114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:14:52.607745 27114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:14:52.616988 27114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:36091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:14:52.618271 27114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:14:52.619784 27114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:14:52.633098 27121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:52.633054 27124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:14:52.633131 27122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:14:52.633805 27114 server_base.cc:1034] running on GCE node
I20260809 15:14:52.634397 27114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:14:52.635356 27114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:14:52.636592 27114 hybrid_clock.cc:648] HybridClock initialized: now 1786288492636546 us; error 53 us; skew 500 ppm
I20260809 15:14:52.636931 27114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:14:52 dist-test-slave-8w9v krb5kdc[26672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288492, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:14:52.640219 27114 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:14:52.641685 27114 webserver.cc:460] Webserver started at http://127.15.11.131:39207/ using document root <none> and password file <none>
I20260809 15:14:52.642414 27114 fs_manager.cc:359] Metadata directory not provided
I20260809 15:14:52.642516 27114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:14:52.642768 27114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:14:52.645928 27114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "f7433edee69f496980b02ecb4a66b8bc"
format_stamp: "Formatted at 2026-08-09 15:14:52 on dist-test-slave-8w9v"
server_key: "12ada9b3867996b8597dfb6c5574f959"
server_key_iv: "45fe14cf8f567be21b6d132b0cff24d4"
server_key_version: "encryptionkey@0"
I20260809 15:14:52.646725 27114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "f7433edee69f496980b02ecb4a66b8bc"
format_stamp: "Formatted at 2026-08-09 15:14:52 on dist-test-slave-8w9v"
server_key: "12ada9b3867996b8597dfb6c5574f959"
server_key_iv: "45fe14cf8f567be21b6d132b0cff24d4"
server_key_version: "encryptionkey@0"
I20260809 15:14:52.651821 27114 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:14:52.655166 27132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:52.656636 27114 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:14:52.656816 27114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "f7433edee69f496980b02ecb4a66b8bc"
format_stamp: "Formatted at 2026-08-09 15:14:52 on dist-test-slave-8w9v"
server_key: "12ada9b3867996b8597dfb6c5574f959"
server_key_iv: "45fe14cf8f567be21b6d132b0cff24d4"
server_key_version: "encryptionkey@0"
I20260809 15:14:52.657012 27114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:14:52.677919 27114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:14:52.681682 27114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:14:52.681984 27114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:14:52.682004 26936 heartbeater.cc:503] Master 127.15.11.190:36091 was elected leader, sending a full tablet report...
I20260809 15:14:52.683135 27114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:14:52.684979 27114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:14:52.685082 27114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:52.685215 27114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:14:52.685305 27114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:14:52.719810 27114 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:40981
I20260809 15:14:52.719849 27267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:40981 every 8 connection(s)
I20260809 15:14:52.721298 27114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:14:52.728688 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 27114
I20260809 15:14:52.728904 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:14:52.729456 15406 external_mini_cluster.cc:1428] Setting key 38878399ac53bc927357d1467f5ed373
Aug 09 15:14:52 dist-test-slave-8w9v krb5kdc[26672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288492, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:14:52.741389 27271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.723888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:39381 (local address 127.15.11.190:36091)
0809 15:14:52.724376 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:52.724381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:52.725347 (+ 966us) server_negotiation.cc:407] Connection header received
0809 15:14:52.727216 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:52.727222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:52.727305 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:52.727430 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:52.729254 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.730257 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.732269 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.732766 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.732867 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.736250 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:52.736255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:52.736260 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:52.736303 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:52.738745 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:52.739402 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:52.739405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:52.739407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:52.739543 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:52.739915 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:52.739918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:52.739920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:52.740249 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:52.740373 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:52.740979 (+ 606us) server_negotiation.cc:299] Negotiation successful
0809 15:14:52.741133 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":269,"threads_started":1}
I20260809 15:14:52.742230 27272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.724470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:36091 (local address 127.15.11.131:39381)
0809 15:14:52.725192 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:52.725239 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:52.726951 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:52.727656 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:52.727664 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:52.728220 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:14:52.729044 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.729073 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.730425 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.730429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.732070 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.732082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.732963 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.732968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.733186 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.733874 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:14:52.733899 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:14:52.735961 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:14:52.738880 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:52.738886 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:52.738899 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:52.739286 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:52.739635 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:14:52.739638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:14:52.739640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:14:52.739805 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:14:52.740355 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:14:52.740361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:14:52.740646 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 15:14:52.740855 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 15:14:52.741206 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":290,"threads_started":1}
I20260809 15:14:52.743702 27268 heartbeater.cc:347] Connected to a master server at 127.15.11.190:36091
I20260809 15:14:52.744041 27268 heartbeater.cc:464] Registering TS with master...
I20260809 15:14:52.744902 27268 heartbeater.cc:511] Master 127.15.11.190:36091 requested a full tablet report, sending...
I20260809 15:14:52.746330 26721 ts_manager.cc:194] Registered new tserver with Master: f7433edee69f496980b02ecb4a66b8bc (127.15.11.131:40981)
I20260809 15:14:52.747067 26721 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:39381
I20260809 15:14:52.756880 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:14:52.774202 27271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.761412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36298 (local address 127.15.11.190:36091)
0809 15:14:52.761631 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:52.761640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:52.762042 (+ 402us) server_negotiation.cc:407] Connection header received
0809 15:14:52.762171 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:52.762174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:52.762237 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:52.762314 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:52.763808 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.764995 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.766208 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.766618 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.766709 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.768921 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:52.768926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:52.768929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:52.768959 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:52.771713 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:52.772417 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:52.772425 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:52.772429 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:52.772570 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:52.772898 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:52.772902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:52.772903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:52.773204 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:52.773424 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:52.773717 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 15:14:52.773881 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:14:52.778071 26721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:14:52.785336 26721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:14:52.809700 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.798500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.15.11.129:41907)
0809 15:14:52.800985 (+ 2485us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:52.800996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:52.801598 (+ 602us) server_negotiation.cc:407] Connection header received
0809 15:14:52.801870 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:52.801877 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:52.802114 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:52.802293 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:52.803356 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.804855 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.807091 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.808450 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.808593 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.808829 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:52.809150 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 15:14:52.809327 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2317,"thread_start_us":2174,"threads_started":1}
I20260809 15:14:52.809711 27287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.798199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41907 (local address 127.0.0.1:53294)
0809 15:14:52.801486 (+ 3287us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:52.801508 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:52.801693 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:52.802531 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:52.802538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:52.802580 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:52.803158 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.803171 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.805028 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.805033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.806925 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.806940 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.808662 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.808669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.808878 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.808953 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 15:14:52.809082 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 15:14:52.809264 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3161,"mutex_wait_us":98,"thread_start_us":879,"threads_started":1}
I20260809 15:14:52.810739 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.798650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.15.11.130:38211)
0809 15:14:52.800471 (+ 1821us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:52.800482 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:52.801486 (+ 1004us) server_negotiation.cc:407] Connection header received
0809 15:14:52.801601 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:52.801608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:52.801882 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:52.802069 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:52.803144 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.804638 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.807616 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.809134 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.809280 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.809464 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:52.810259 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 15:14:52.810442 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1603,"thread_start_us":1458,"threads_started":1}
I20260809 15:14:52.810772 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.797532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:38211 (local address 127.0.0.1:42006)
0809 15:14:52.801147 (+ 3615us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:52.801191 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:52.801352 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:52.802362 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:52.802369 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:52.802403 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:52.802913 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.802929 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.804798 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.804804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.807451 (+ 2647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.807465 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.809625 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.809632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.810068 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.810094 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 15:14:52.810203 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 15:14:52.810327 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3469,"mutex_wait_us":121,"thread_start_us":3660,"threads_started":1}
I20260809 15:14:52.811579 27288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.797225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:40981 (local address 127.0.0.1:54000)
0809 15:14:52.799559 (+ 2334us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:52.799603 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:52.799796 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:52.803609 (+ 3813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:52.803615 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:52.803640 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:52.804007 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.804023 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.805879 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.805888 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.807986 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:52.808002 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.810778 (+ 2776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.810785 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:52.810978 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.811010 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:14:52.811082 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 15:14:52.811202 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2015,"mutex_wait_us":45,"thread_start_us":789,"threads_started":1}
I20260809 15:14:52.812148 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:52.797442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54000 (local address 127.15.11.131:40981)
0809 15:14:52.803037 (+ 5595us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:52.803048 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:52.803080 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:14:52.803178 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:52.803183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:52.803404 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:52.803599 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:52.804224 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.805736 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.808200 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:52.810491 (+ 2291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:52.811337 (+ 846us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:52.811573 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:52.811637 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 15:14:52.811788 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5393,"thread_start_us":5243,"threads_started":1}
I20260809 15:14:52.814280 27019 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3d0e269a9242699f571c58002a7ec1 (DEFAULT_TABLE table=test-table [id=a6abee837d9546229e84af650f5ad547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:52.814121 26861 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3d0e269a9242699f571c58002a7ec1 (DEFAULT_TABLE table=test-table [id=a6abee837d9546229e84af650f5ad547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:52.815459 27019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3d0e269a9242699f571c58002a7ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:52.815604 26861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3d0e269a9242699f571c58002a7ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:52.815804 27189 tablet_service.cc:1467] Processing CreateTablet for tablet 0c3d0e269a9242699f571c58002a7ec1 (DEFAULT_TABLE table=test-table [id=a6abee837d9546229e84af650f5ad547]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:14:52.817166 27189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c3d0e269a9242699f571c58002a7ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:14:52.827275 27292 tablet_bootstrap.cc:492] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: Bootstrap starting.
I20260809 15:14:52.830070 27294 tablet_bootstrap.cc:492] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: Bootstrap starting.
I20260809 15:14:52.831179 27292 tablet_bootstrap.cc:654] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:52.832612 27292 log.cc:824] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:52.832867 27296 tablet_bootstrap.cc:492] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: Bootstrap starting.
I20260809 15:14:52.834106 27294 tablet_bootstrap.cc:654] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:52.835670 27296 tablet_bootstrap.cc:654] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: Neither blocks nor log segments found. Creating new log.
I20260809 15:14:52.835633 27294 log.cc:824] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:52.836956 27296 log.cc:824] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: Log is configured to *not* fsync() on all Append() calls
I20260809 15:14:52.838584 27292 tablet_bootstrap.cc:492] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: No bootstrap required, opened a new log
I20260809 15:14:52.838689 27294 tablet_bootstrap.cc:492] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: No bootstrap required, opened a new log
I20260809 15:14:52.838886 27292 ts_tablet_manager.cc:1397] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.006s
I20260809 15:14:52.838943 27294 ts_tablet_manager.cc:1397] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 15:14:52.839916 27296 tablet_bootstrap.cc:492] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: No bootstrap required, opened a new log
I20260809 15:14:52.840207 27296 ts_tablet_manager.cc:1397] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260809 15:14:52.846329 27292 raft_consensus.cc:348] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } }
I20260809 15:14:52.846678 27292 raft_consensus.cc:374] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:52.846809 27292 raft_consensus.cc:729] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f4c0804512e4b03b9d87361101fc4e1, State: Initialized, Role: FOLLOWER
I20260809 15:14:52.847447 27292 consensus_queue.cc:260] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } }
I20260809 15:14:52.848562 27105 heartbeater.cc:503] Master 127.15.11.190:36091 was elected leader, sending a full tablet report...
I20260809 15:14:52.849234 27292 ts_tablet_manager.cc:1428] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20260809 15:14:52.849498 27294 raft_consensus.cc:348] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } }
I20260809 15:14:52.849965 27294 raft_consensus.cc:374] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:52.850118 27294 raft_consensus.cc:729] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27128422655744cd9cf2d31705005420, State: Initialized, Role: FOLLOWER
I20260809 15:14:52.850716 27294 consensus_queue.cc:260] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } }
I20260809 15:14:52.851858 27296 raft_consensus.cc:348] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } }
I20260809 15:14:52.852408 27296 raft_consensus.cc:374] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:14:52.852483 27294 ts_tablet_manager.cc:1428] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260809 15:14:52.852550 27296 raft_consensus.cc:729] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7433edee69f496980b02ecb4a66b8bc, State: Initialized, Role: FOLLOWER
I20260809 15:14:52.853283 27296 consensus_queue.cc:260] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } }
I20260809 15:14:52.854359 27268 heartbeater.cc:503] Master 127.15.11.190:36091 was elected leader, sending a full tablet report...
I20260809 15:14:52.855099 27296 ts_tablet_manager.cc:1428] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
W20260809 15:14:52.899878 26937 tablet.cc:2369] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:14:52.935784 27106 tablet.cc:2369] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:14:52.973021 27270 tablet.cc:2369] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:14:53.040294 27303 raft_consensus.cc:482] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:14:53.040670 27303 raft_consensus.cc:504] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } }
I20260809 15:14:53.042412 27303 leader_election.cc:288] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7433edee69f496980b02ecb4a66b8bc (127.15.11.131:40981), 27128422655744cd9cf2d31705005420 (127.15.11.129:41907)
I20260809 15:14:53.051100 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:53.042918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:58841 (local address 127.15.11.131:40981)
0809 15:14:53.043158 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:53.043167 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:53.044104 (+ 937us) server_negotiation.cc:407] Connection header received
0809 15:14:53.044330 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:53.044336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:53.044420 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:53.044489 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:53.045209 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.046182 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.048700 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.049945 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.050112 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:53.050185 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:53.050694 (+ 509us) server_negotiation.cc:299] Negotiation successful
0809 15:14:53.050828 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:14:53.051103 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:53.043003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:37895 (local address 127.15.11.129:41907)
0809 15:14:53.043419 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:53.043428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:53.043745 (+ 317us) server_negotiation.cc:407] Connection header received
0809 15:14:53.044004 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:53.044008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:53.044098 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:53.044215 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:14:53.044840 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.045843 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.048696 (+ 2853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.049946 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.050112 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:53.050185 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:53.050694 (+ 509us) server_negotiation.cc:299] Negotiation successful
0809 15:14:53.050828 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260809 15:14:53.051398 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:53.042703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41907 (local address 127.15.11.130:37895)
0809 15:14:53.043622 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:53.043637 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:53.043839 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:53.044231 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:53.044238 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:53.044266 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:53.044650 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:53.044663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.045990 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.045994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:53.048526 (+ 2532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:53.048543 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.050251 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.050258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:53.050493 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:53.050541 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 15:14:53.050641 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 15:14:53.050844 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":70,"thread_start_us":562,"threads_started":1}
I20260809 15:14:53.051431 27307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:53.042927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:40981 (local address 127.15.11.130:58841)
0809 15:14:53.043990 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0809 15:14:53.044005 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:14:53.044178 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:14:53.044548 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:14:53.044553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:14:53.044585 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:14:53.045052 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:53.045062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.046340 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.046345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:53.048535 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:14:53.048543 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.050266 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.050272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:14:53.050487 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:53.050550 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 15:14:53.050641 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:14:53.050839 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":950,"mutex_wait_us":91,"thread_start_us":739,"threads_started":1}
I20260809 15:14:53.052547 27214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3d0e269a9242699f571c58002a7ec1" candidate_uuid: "5f4c0804512e4b03b9d87361101fc4e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7433edee69f496980b02ecb4a66b8bc" is_pre_election: true
I20260809 15:14:53.052551 26884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3d0e269a9242699f571c58002a7ec1" candidate_uuid: "5f4c0804512e4b03b9d87361101fc4e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27128422655744cd9cf2d31705005420" is_pre_election: true
I20260809 15:14:53.053071 27214 raft_consensus.cc:2454] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4c0804512e4b03b9d87361101fc4e1 in term 0.
I20260809 15:14:53.053071 26884 raft_consensus.cc:2454] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f4c0804512e4b03b9d87361101fc4e1 in term 0.
I20260809 15:14:53.053762 26964 leader_election.cc:302] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 27128422655744cd9cf2d31705005420, 5f4c0804512e4b03b9d87361101fc4e1, f7433edee69f496980b02ecb4a66b8bc; no voters:
I20260809 15:14:53.054186 27303 raft_consensus.cc:2781] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:14:53.054340 27303 raft_consensus.cc:482] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:14:53.054458 27303 raft_consensus.cc:3037] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:53.056852 27303 raft_consensus.cc:504] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } }
I20260809 15:14:53.057490 27303 leader_election.cc:288] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [CANDIDATE]: Term 1 election: Requested vote from peers f7433edee69f496980b02ecb4a66b8bc (127.15.11.131:40981), 27128422655744cd9cf2d31705005420 (127.15.11.129:41907)
I20260809 15:14:53.057964 27214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3d0e269a9242699f571c58002a7ec1" candidate_uuid: "5f4c0804512e4b03b9d87361101fc4e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7433edee69f496980b02ecb4a66b8bc"
I20260809 15:14:53.057989 26884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c3d0e269a9242699f571c58002a7ec1" candidate_uuid: "5f4c0804512e4b03b9d87361101fc4e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27128422655744cd9cf2d31705005420"
I20260809 15:14:53.058179 27214 raft_consensus.cc:3037] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:53.058197 26884 raft_consensus.cc:3037] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:14:53.060544 26884 raft_consensus.cc:2454] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f4c0804512e4b03b9d87361101fc4e1 in term 1.
I20260809 15:14:53.060865 27214 raft_consensus.cc:2454] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f4c0804512e4b03b9d87361101fc4e1 in term 1.
I20260809 15:14:53.061046 26965 leader_election.cc:302] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27128422655744cd9cf2d31705005420, 5f4c0804512e4b03b9d87361101fc4e1; no voters:
I20260809 15:14:53.061386 27303 raft_consensus.cc:2781] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:14:53.061868 27303 raft_consensus.cc:686] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 1 LEADER]: Becoming Leader. State: Replica: 5f4c0804512e4b03b9d87361101fc4e1, State: Running, Role: LEADER
I20260809 15:14:53.062326 27303 consensus_queue.cc:237] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } }
I20260809 15:14:53.066768 26721 catalog_manager.cc:5495] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f4c0804512e4b03b9d87361101fc4e1 (127.15.11.130). New cstate: current_term: 1 leader_uuid: "5f4c0804512e4b03b9d87361101fc4e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f4c0804512e4b03b9d87361101fc4e1" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 } health_report { overall_health: UNKNOWN } } }
I20260809 15:14:53.103236 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:53.098225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.15.11.130:38211)
0809 15:14:53.098421 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:53.098425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:53.098509 (+ 84us) server_negotiation.cc:407] Connection header received
0809 15:14:53.098674 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:53.098678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:53.098754 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:53.098896 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:14:53.099220 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.099992 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.101314 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.101715 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.101807 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:53.102227 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:14:53.102765 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:14:53.102894 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:53.102954 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 15:14:53.103018 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:14:53.111932 26884 raft_consensus.cc:1261] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [term 1 FOLLOWER]: Refusing update from remote peer 5f4c0804512e4b03b9d87361101fc4e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:53.111923 27214 raft_consensus.cc:1261] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [term 1 FOLLOWER]: Refusing update from remote peer 5f4c0804512e4b03b9d87361101fc4e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:14:53.113121 27308 consensus_queue.cc:1035] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27128422655744cd9cf2d31705005420" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:53.113487 27303 consensus_queue.cc:1035] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7433edee69f496980b02ecb4a66b8bc" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:14:53.125062 27311 mvcc.cc:205] Tried to move back new op lower bound from 7316637667774308352 to 7316637667587129344. Current Snapshot: MvccSnapshot[applied={T|T < 7316637667774308352}]
I20260809 15:14:53.142303 26721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:53.142612 26721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:14:53.146416 26721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11a289e9cabf499ead68b7f15e8c0606: Sending DeleteTablet for 3 replicas of tablet 0c3d0e269a9242699f571c58002a7ec1
I20260809 15:14:53.147636 27189 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c3d0e269a9242699f571c58002a7ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:53 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:54000
I20260809 15:14:53.147616 27019 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c3d0e269a9242699f571c58002a7ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:53 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:42006
I20260809 15:14:53.148691 26861 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c3d0e269a9242699f571c58002a7ec1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:14:53 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:53294
I20260809 15:14:53.149638 27324 tablet_replica.cc:330] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: stopping tablet replica
I20260809 15:14:53.150106 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 26790
I20260809 15:14:53.150108 27324 raft_consensus.cc:2229] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:53.150621 27324 raft_consensus.cc:2258] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:53.152904 27324 ts_tablet_manager.cc:1905] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:53.160379 27324 ts_tablet_manager.cc:1918] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:53.160769 27324 log.cc:1196] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/0c3d0e269a9242699f571c58002a7ec1
I20260809 15:14:53.161420 27324 ts_tablet_manager.cc:1939] T 0c3d0e269a9242699f571c58002a7ec1 P 27128422655744cd9cf2d31705005420: Deleting consensus metadata
I20260809 15:14:53.163633 26706 catalog_manager.cc:4841] TS 27128422655744cd9cf2d31705005420 (127.15.11.129:41907): tablet 0c3d0e269a9242699f571c58002a7ec1 (table test-table [id=a6abee837d9546229e84af650f5ad547]) successfully deleted
I20260809 15:14:53.164755 27322 tablet_replica.cc:330] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: stopping tablet replica
I20260809 15:14:53.165068 27321 tablet_replica.cc:330] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: stopping tablet replica
I20260809 15:14:53.165395 27322 raft_consensus.cc:2229] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:14:53.165632 27321 raft_consensus.cc:2229] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:14:53.165983 27322 raft_consensus.cc:2258] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:53.166402 27321 raft_consensus.cc:2258] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:14:53.168407 27322 ts_tablet_manager.cc:1905] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:14:53.168614 27321 ts_tablet_manager.cc:1905] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 15:14:53 dist-test-slave-8w9v krb5kdc[26672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:14:53.176579 27321 ts_tablet_manager.cc:1918] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:53.176857 27321 log.cc:1196] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/0c3d0e269a9242699f571c58002a7ec1
I20260809 15:14:53.177510 27321 ts_tablet_manager.cc:1939] T 0c3d0e269a9242699f571c58002a7ec1 P 5f4c0804512e4b03b9d87361101fc4e1: Deleting consensus metadata
I20260809 15:14:53.177776 27322 ts_tablet_manager.cc:1918] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:14:53.178007 27322 log.cc:1196] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/0c3d0e269a9242699f571c58002a7ec1
I20260809 15:14:53.178593 27322 ts_tablet_manager.cc:1939] T 0c3d0e269a9242699f571c58002a7ec1 P f7433edee69f496980b02ecb4a66b8bc: Deleting consensus metadata
I20260809 15:14:53.180539 26705 catalog_manager.cc:4841] TS f7433edee69f496980b02ecb4a66b8bc (127.15.11.131:40981): tablet 0c3d0e269a9242699f571c58002a7ec1 (table test-table [id=a6abee837d9546229e84af650f5ad547]) successfully deleted
I20260809 15:14:53.182061 26704 catalog_manager.cc:4841] TS 5f4c0804512e4b03b9d87361101fc4e1 (127.15.11.130:38211): tablet 0c3d0e269a9242699f571c58002a7ec1 (table test-table [id=a6abee837d9546229e84af650f5ad547]) successfully deleted
I20260809 15:14:53.182916 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:53.151981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53308 (local address 127.15.11.129:41907)
0809 15:14:53.152226 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:53.152235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:53.158782 (+ 6547us) server_negotiation.cc:407] Connection header received
0809 15:14:53.159777 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:53.159785 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:53.159904 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:53.160018 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:53.164301 (+ 4283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.165554 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.167432 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.168034 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.168170 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:53.173712 (+ 5542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:53.173727 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:53.173745 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:53.173795 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:53.178298 (+ 4503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:53.180274 (+ 1976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:53.180283 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:53.180286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:53.180477 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:53.180952 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:53.180959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:53.180962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:53.181413 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:53.181686 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:53.182276 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 15:14:53.182545 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:14:53.183988 26931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:53.281818 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 26790
I20260809 15:14:53.303485 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 26943
Aug 09 15:14:53 dist-test-slave-8w9v krb5kdc[26672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:14:53.325425 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:53.304374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42026 (local address 127.15.11.130:38211)
0809 15:14:53.304573 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:53.304579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:53.304602 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:14:53.304748 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:53.304754 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:53.304843 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:53.304976 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:53.306169 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.307478 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.308734 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.309401 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.309542 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:53.319284 (+ 9742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:53.319299 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:53.319315 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:53.319350 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:53.322620 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:53.323436 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:53.323443 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:53.323445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:53.323581 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:53.323967 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:53.323970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:53.323972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:53.324369 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:53.324616 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:53.324935 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 15:14:53.325100 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:14:53.326200 27098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:53.440743 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 26943
I20260809 15:14:53.463893 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 27114
Aug 09 15:14:53 dist-test-slave-8w9v krb5kdc[26672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:14:53.479173 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:14:53.464724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36954 (local address 127.15.11.131:40981)
0809 15:14:53.464912 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 15:14:53.464918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:14:53.464967 (+ 49us) server_negotiation.cc:407] Connection header received
0809 15:14:53.465189 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:14:53.465198 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:14:53.465257 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:14:53.465354 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:14:53.466621 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.467334 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.468799 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:14:53.469188 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:14:53.469321 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:14:53.473839 (+ 4518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:14:53.473853 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:14:53.473870 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:14:53.473916 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:14:53.476817 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:53.477407 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:53.477414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:53.477416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:53.477541 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:14:53.477842 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:14:53.477845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:14:53.477847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:14:53.478199 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:14:53.478412 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 15:14:53.478611 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 15:14:53.478776 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:14:53.479856 27263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:53.567230 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 27114
I20260809 15:14:53.587476 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 26687
I20260809 15:14:53.588711 26754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:14:53.698426 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 26687
2026-08-09T15:14:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4519 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:14:53 dist-test-slave-8w9v krb5kdc[27332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:14:53 dist-test-slave-8w9v krb5kdc[27332](info): set up 2 sockets
Aug 09 15:14:53 dist-test-slave-8w9v krb5kdc[27332](info): commencing operation
krb5kdc: starting...
W20260809 15:14:54.853174 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:14:55 dist-test-slave-8w9v krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:14:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/rangeradmin_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/rangeradmin_127.15.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/rangerlookup_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/rangerlookup_127.15.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/HTTP_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/HTTP_127.15.11.148@KRBTEST.COM.keytab.
I20260809 15:14:55.214538 15406 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFizLz9/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/postgres -l logfile start
2026-08-09 15:14:57.738 UTC [27366] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 15:14:57.738 UTC [27366] LOG: listening on IPv4 address "127.15.11.148", port 43863
2026-08-09 15:14:57.741 UTC [27366] LOG: listening on Unix socket "/tmp/.s.PGSQL.43863"
2026-08-09 15:14:57.766 UTC [27369] LOG: database system was shut down at 2026-08-09 15:14:56 UTC
2026-08-09 15:14:57.770 UTC [27366] LOG: database system is ready to accept connections
I20260809 15:14:58.802817 15406 mini_postgres.cc:96] Postgres bound to 43863
2026-08-09 15:14:58.826 UTC [27379] FATAL: database "slave" does not exist
127.15.11.148:43863 - accepting connections
I20260809 15:14:58.827603 15406 mini_ranger.cc:161] Starting Ranger...
I20260809 15:14:58.856879 15406 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 15:14:59.311407 15406 mini_ranger.cc:87] Created ranger Postgres database
I20260809 15:14:59.311551 15406 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-admin
2026-08-09 15:14:59,768 [I] DB FLAVOR :POSTGRES
2026-08-09 15:14:59,768 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:14:59,768 [I] Checking connection..
2026-08-09 15:14:59,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 15:15:00,047 [I] Checking connection passed.
2026-08-09 15:15:00,048 [I] --------- Verifying version history table ---------
2026-08-09 15:15:00,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:15:00,308 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:15:00,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:15:00,572 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:15:00,572 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 15:15:00,572 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 15:15:00.829 UTC [27458] WARNING: there is no transaction in progress
2026-08-09 15:15:00,836 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 15:15:00,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:15:01,069 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 15:15:01,070 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:15:01,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 15:15:01,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 15:15:01,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8w9v.c.gcp-upstream.internal','N') ;"
2026-08-09 15:15:01,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:15:02,058 [I] Table x_portal_user does not exist in database ranger
2026-08-09 15:15:02,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:15:02,330 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 15:15:02,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:15:02,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:15:02,861 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 15:15:02,861 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 15:15:03.780 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.796 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.811 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.825 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.844 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.862 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.927 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.933 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.947 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.957 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.967 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.980 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.989 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:03.995 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:04.001 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:04.008 UTC [27579] WARNING: there is no transaction in progress
2026-08-09 15:15:04,607 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 15:15:04,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 15:15:04,858 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 15:15:04,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:15:05,109 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 15:15:05,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:15:05,366 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 15:15:05,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:15:05,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:15:05,886 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8w9v.c.gcp-upstream.internal';"
2026-08-09 15:15:06,101 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 15:15:06,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 15:15:06,361 [I] DB_PATCHES have already been applied
I20260809 15:15:06.367123 15406 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFizLz9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 15:15:06.367215 15406 mini_ranger.cc:193] Using host: 127.15.11.148
I20260809 15:15:06.387850 15406 mini_ranger.cc:227] Ranger admin URL: http://127.15.11.148:54507
Aug 09, 2026 3:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:15:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:15:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:15:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 15:15:07.576970 15406 mini_ranger.cc:160] Time spent starting Ranger: real 8.749s user 0.005s sys 0.048s
Aug 09, 2026 3:15:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:15:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 3:15:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:15:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:15:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 15:15:30.019176 15406 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/rangerkms_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/rangerkms_127.15.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.11.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.11.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.11.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/HTTP_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.11.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/HTTP_127.15.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 15:15:30 dist-test-slave-8w9v krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288530, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 15:15:30.197201 15406 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 15:15:30.218632 15406 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 15:15:30.928941 15406 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 15:15:30.929117 15406 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-kms
I20260809 15:15:30.929253 15406 mini_ranger_kms.cc:227] Using postgres at 127.15.11.148:43863
2026-08-09 15:15:31,581 [I] DB FLAVOR :POSTGRES
2026-08-09 15:15:31,581 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:15:31,581 [I] Checking connection
2026-08-09 15:15:31,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 15:15:31,821 [I] connection success
2026-08-09 15:15:31,822 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 15:15:31,822 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 15:15:31,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 15:15:32,066 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 15:15:32,066 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:15:32,067 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 15:15:32,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:43863/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 15:15:32,312 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 15:15:32.781126 15406 mini_ranger_kms.cc:310] Created kms service
I20260809 15:15:32.955760 15406 mini_ranger_kms.cc:326] Created kudu user
I20260809 15:15:32.998909 15406 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 15:15:32 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288530, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:15:33.391731 15406 mini_ranger_kms.cc:384] Added ranger policy
I20260809 15:15:33.392051 15406 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 15:15:33.392117 15406 mini_ranger_kms.cc:240] Using host: 127.15.11.148
I20260809 15:15:33.416648 15406 mini_ranger_kms.cc:276] Ranger KMS PID: 27780
I20260809 15:15:33.416862 15406 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.11.148:57121
Aug 09, 2026 3:15:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:15:34 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:15:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:15:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:15:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 15:15:35.806237 15406 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.609s user 0.129s sys 0.411s
I20260809 15:15:35.806506 15406 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 15:15:35.806743 15406 mini_ranger_kms.cc:401] 127.15.11.148:57121/kms/v1/keys
Aug 09 15:15:35 dist-test-slave-8w9v krb5kdc[27332](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288535, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.11.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 15:15:36 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288535, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.11.148@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
Aug 09 15:15:36 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288535, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.11.148@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
Aug 09 15:15:38 dist-test-slave-8w9v krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:15:38.891481 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37707
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:37707
--ranger_config_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:57121/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:15:39.229046 27833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:15:39.229316 27833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:15:39.229398 27833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:15:39.236282 27833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:15:39.236407 27833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:15:39.236456 27833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:15:39.236511 27833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:15:39.236559 27833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:15:39.246802 27833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.11.148:57121/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:37707
--ranger_config_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:15:39.248066 27833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:15:39.249348 27833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:15:39.261165 27839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:15:39.261161 27838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:15:39.261432 27841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:15:39.262343 27833 server_base.cc:1034] running on GCE node
I20260809 15:15:39.263083 27833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:15:39.264452 27833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:15:39.265717 27833 hybrid_clock.cc:648] HybridClock initialized: now 1786288539265630 us; error 54 us; skew 500 ppm
I20260809 15:15:39.266078 27833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:15:39 dist-test-slave-8w9v krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:15:39.274428 27833 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:15:39.275868 27833 webserver.cc:460] Webserver started at http://127.15.11.190:34069/ using document root <none> and password file <none>
I20260809 15:15:39.276585 27833 fs_manager.cc:359] Metadata directory not provided
I20260809 15:15:39.276726 27833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:15:39.277050 27833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:15:39 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288539, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:15:39.337944 27833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "cc47db25939a43f3bcd594e247886c7f"
format_stamp: "Formatted at 2026-08-09 15:15:39 on dist-test-slave-8w9v"
server_key: "82879b1bcfc8a03ed1c9be9ca48674c2"
server_key_iv: "6d2507c14c634c64ffa0a304c027a04a"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:39.339003 27833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "cc47db25939a43f3bcd594e247886c7f"
format_stamp: "Formatted at 2026-08-09 15:15:39 on dist-test-slave-8w9v"
server_key: "82879b1bcfc8a03ed1c9be9ca48674c2"
server_key_iv: "6d2507c14c634c64ffa0a304c027a04a"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:39.346127 27833 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260809 15:15:39.365533 27851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:39.367254 27833 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:15:39.367451 27833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "cc47db25939a43f3bcd594e247886c7f"
format_stamp: "Formatted at 2026-08-09 15:15:39 on dist-test-slave-8w9v"
server_key: "82879b1bcfc8a03ed1c9be9ca48674c2"
server_key_iv: "6d2507c14c634c64ffa0a304c027a04a"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:39.367729 27833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:15:39.391060 27833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:15:39.395624 27833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:15:39.395934 27833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:15:39.415208 27908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:37707 every 8 connection(s)
I20260809 15:15:39.415206 27833 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:37707
I20260809 15:15:39.416929 27833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:15:39.420770 27910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:15:39.423491 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 27833
I20260809 15:15:39.423683 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:15:39.424584 15406 external_mini_cluster.cc:1428] Setting key a8adb131e5e28a14fbe394b68eac5ee8
I20260809 15:15:39.436600 27910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f: Bootstrap starting.
I20260809 15:15:39.441699 27910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f: Neither blocks nor log segments found. Creating new log.
I20260809 15:15:39.442970 27910 log.cc:824] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:15:39.446466 27910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f: No bootstrap required, opened a new log
Aug 09 15:15:39 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:15:39.456425 27910 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc47db25939a43f3bcd594e247886c7f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37707 } }
I20260809 15:15:39.456933 27910 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:15:39.457068 27910 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc47db25939a43f3bcd594e247886c7f, State: Initialized, Role: FOLLOWER
I20260809 15:15:39.457757 27910 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc47db25939a43f3bcd594e247886c7f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37707 } }
I20260809 15:15:39.458077 27910 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:15:39.458243 27910 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:15:39.458487 27910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:15:39.461056 27910 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc47db25939a43f3bcd594e247886c7f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37707 } }
I20260809 15:15:39.461740 27910 leader_election.cc:302] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc47db25939a43f3bcd594e247886c7f; no voters:
I20260809 15:15:39.462494 27910 leader_election.cc:288] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:15:39.462584 27917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:15:39.466655 27917 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [term 1 LEADER]: Becoming Leader. State: Replica: cc47db25939a43f3bcd594e247886c7f, State: Running, Role: LEADER
I20260809 15:15:39.467280 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:39.426809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34880 (local address 127.15.11.190:37707)
0809 15:15:39.441709 (+ 14900us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:39.441724 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:39.441771 (+ 47us) server_negotiation.cc:407] Connection header received
0809 15:15:39.444301 (+ 2530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:39.444330 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:39.444775 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:39.445163 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:15:39.446729 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:39.448868 (+ 2139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:39.450625 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:39.451695 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:39.451887 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:39.456937 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:15:39.456948 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:15:39.456966 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:15:39.457015 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:15:39.461011 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:39.461862 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:39.461870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:39.461879 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:39.462059 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:39.462628 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:39.462634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:39.462637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:39.463479 (+ 842us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:15:39.463873 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:39.464956 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0809 15:15:39.465476 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14646,"thread_start_us":14435,"threads_started":1}
I20260809 15:15:39.467319 27917 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc47db25939a43f3bcd594e247886c7f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37707 } }
I20260809 15:15:39.471777 27910 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:15:39.477730 27918 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc47db25939a43f3bcd594e247886c7f. Latest consensus state: current_term: 1 leader_uuid: "cc47db25939a43f3bcd594e247886c7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc47db25939a43f3bcd594e247886c7f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37707 } } }
I20260809 15:15:39.478641 27918 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [sys.catalog]: This master's current role is: LEADER
I20260809 15:15:39.478314 27919 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc47db25939a43f3bcd594e247886c7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc47db25939a43f3bcd594e247886c7f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37707 } } }
I20260809 15:15:39.479156 27919 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f [sys.catalog]: This master's current role is: LEADER
I20260809 15:15:39.485136 27910 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 15:15:39.495008 27923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:15:39.503242 27923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:15:39.527251 27923 catalog_manager.cc:1347] Generated new cluster ID: 1322a572d47044f5a56781f435e9c7d5
I20260809 15:15:39.528685 27923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:15:39.567833 27923 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:15:39.570188 27923 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:15:39.583530 27923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f: Generated new TSK 0
I20260809 15:15:39.585091 27923 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:15:40.390901 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:57121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37707
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:15:40 dist-test-slave-8w9v krb5kdc[27332](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288540, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 15:15:40.831061 27962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:15:40.831375 27962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:15:40.831581 27962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:15:40.840886 27962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:15:40.841040 27962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:15:40.841233 27962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:15:40.851820 27962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.11.148:57121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:15:40.853222 27962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:15:40.854529 27962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:15:40.873499 27972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:15:40.874430 27973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:15:40.881893 27977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:15:40.882545 27962 server_base.cc:1034] running on GCE node
I20260809 15:15:40.883562 27962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:15:40.896428 27962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:15:40.899904 27962 hybrid_clock.cc:648] HybridClock initialized: now 1786288540899769 us; error 114 us; skew 500 ppm
I20260809 15:15:40.900394 27962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:15:40 dist-test-slave-8w9v krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288540, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:15:40.904274 27962 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:15:40.907709 27962 webserver.cc:460] Webserver started at http://127.15.11.129:41931/ using document root <none> and password file <none>
I20260809 15:15:40.908701 27962 fs_manager.cc:359] Metadata directory not provided
I20260809 15:15:40.908871 27962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:15:40.909251 27962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:15:40 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288540, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:15:40.929423 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4"
format_stamp: "Formatted at 2026-08-09 15:15:40 on dist-test-slave-8w9v"
server_key: "808267ef95f5ebcae501917c735007b6"
server_key_iv: "bf9d913d9bbe55d012da4bf6b17beaa6"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:40.930548 27962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4"
format_stamp: "Formatted at 2026-08-09 15:15:40 on dist-test-slave-8w9v"
server_key: "808267ef95f5ebcae501917c735007b6"
server_key_iv: "bf9d913d9bbe55d012da4bf6b17beaa6"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:40.937979 27962 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260809 15:15:40.959028 27984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:40.960979 27962 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:15:40.961234 27962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4"
format_stamp: "Formatted at 2026-08-09 15:15:40 on dist-test-slave-8w9v"
server_key: "808267ef95f5ebcae501917c735007b6"
server_key_iv: "bf9d913d9bbe55d012da4bf6b17beaa6"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:40.961601 27962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:15:41.033728 27962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:15:41.039407 27962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:15:41.039925 27962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:15:41.041129 27962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:15:41.042712 27962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:15:41.042815 27962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:41.042960 27962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:15:41.043038 27962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:41.132879 27962 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:39097
I20260809 15:15:41.132895 28108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:39097 every 8 connection(s)
I20260809 15:15:41.134495 27962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:15:41.140141 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 27962
I20260809 15:15:41.140400 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:15:41.141511 15406 external_mini_cluster.cc:1428] Setting key aaa84dc5bfdfc1e0cf2bbb56597a2d9c
Aug 09 15:15:41 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288540, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
Aug 09 15:15:41 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288540, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
I20260809 15:15:41.190476 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:41.138782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:48361 (local address 127.15.11.190:37707)
0809 15:15:41.139455 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:41.139465 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:41.152703 (+ 13238us) server_negotiation.cc:407] Connection header received
0809 15:15:41.155702 (+ 2999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:41.155709 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:41.155842 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:41.155943 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:15:41.158602 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:41.159837 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:41.162187 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:41.162919 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:41.163035 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:41.167934 (+ 4899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:15:41.167942 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:15:41.167947 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:15:41.168010 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:15:41.171923 (+ 3913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:41.186756 (+ 14833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:41.186763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:41.186767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:41.187175 (+ 408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:41.188173 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:41.188178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:41.188181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:41.188576 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:15:41.188718 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:41.189791 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0809 15:15:41.190059 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":423,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
I20260809 15:15:41.204707 28112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:41.138742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37707 (local address 127.15.11.129:48361)
0809 15:15:41.152488 (+ 13746us) negotiation.cc:107] Waiting for socket to connect
0809 15:15:41.152557 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 15:15:41.155318 (+ 2761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:15:41.156667 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:15:41.156681 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:15:41.157400 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:15:41.158361 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:41.158392 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:41.160090 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:41.160113 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:41.161989 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:41.162006 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:41.163462 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:41.163472 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:41.163781 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:41.164597 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:15:41.164634 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:15:41.167634 (+ 3000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:15:41.185820 (+ 18186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:15:41.185831 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:15:41.185853 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:15:41.186589 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:15:41.187458 (+ 869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:15:41.187466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:15:41.187471 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:15:41.187928 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:15:41.188822 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:15:41.188831 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:15:41.189327 (+ 496us) client_negotiation.cc:769] Sending connection context
0809 15:15:41.191056 (+ 1729us) client_negotiation.cc:240] Negotiation successful
0809 15:15:41.191703 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13370,"thread_start_us":13065,"threads_started":1}
I20260809 15:15:41.207000 28109 heartbeater.cc:347] Connected to a master server at 127.15.11.190:37707
I20260809 15:15:41.207440 28109 heartbeater.cc:464] Registering TS with master...
I20260809 15:15:41.208544 28109 heartbeater.cc:511] Master 127.15.11.190:37707 requested a full tablet report, sending...
I20260809 15:15:41.211625 27869 ts_manager.cc:194] Registered new tserver with Master: 3e9a9c07b9e8403e92ba0fc9204ff7f4 (127.15.11.129:39097)
I20260809 15:15:41.214098 27869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:48361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:15:41.244572 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:57121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37707
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:15:41 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288540, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
W20260809 15:15:41.767021 28115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:15:41.767527 28115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:15:41.767701 28115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:15:41.778239 28115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:15:41.778450 28115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:15:41.778690 28115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:15:41.792238 28115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.11.148:57121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:15:41.793957 28115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:15:41.795660 28115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:15:41.814891 28122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:15:41.815596 28123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:15:41.823576 28125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:15:41.824014 28115 server_base.cc:1034] running on GCE node
I20260809 15:15:41.824802 28115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:15:41.825912 28115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:15:41.827186 28115 hybrid_clock.cc:648] HybridClock initialized: now 1786288541827078 us; error 102 us; skew 500 ppm
I20260809 15:15:41.827566 28115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:15:41 dist-test-slave-8w9v krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288541, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:15:41.830904 28115 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:15:41.837641 28115 webserver.cc:460] Webserver started at http://127.15.11.130:44451/ using document root <none> and password file <none>
I20260809 15:15:41.838685 28115 fs_manager.cc:359] Metadata directory not provided
I20260809 15:15:41.838876 28115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:15:41.839301 28115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:15:41 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288541, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:15:41.856422 28115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "c884ec24c02f4b499a6c0ac20e3d0669"
format_stamp: "Formatted at 2026-08-09 15:15:41 on dist-test-slave-8w9v"
server_key: "5dd7a2864cc52e4de5dc192762ff1a27"
server_key_iv: "18538739d196abba52f00af17457f4ab"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:41.858007 28115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "c884ec24c02f4b499a6c0ac20e3d0669"
format_stamp: "Formatted at 2026-08-09 15:15:41 on dist-test-slave-8w9v"
server_key: "5dd7a2864cc52e4de5dc192762ff1a27"
server_key_iv: "18538739d196abba52f00af17457f4ab"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:41.866554 28115 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260809 15:15:41.885210 28142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:41.886847 28115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:15:41.887074 28115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "c884ec24c02f4b499a6c0ac20e3d0669"
format_stamp: "Formatted at 2026-08-09 15:15:41 on dist-test-slave-8w9v"
server_key: "5dd7a2864cc52e4de5dc192762ff1a27"
server_key_iv: "18538739d196abba52f00af17457f4ab"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:41.887332 28115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:15:41.910400 28115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:15:41.914515 28115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:15:41.914832 28115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:15:41.915972 28115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:15:41.917585 28115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:15:41.917695 28115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:41.917876 28115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:15:41.917949 28115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:41.962817 28115 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:44139
I20260809 15:15:41.962836 28280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:44139 every 8 connection(s)
I20260809 15:15:41.964262 28115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:15:41.968978 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 28115
I20260809 15:15:41.969177 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:15:41.969643 15406 external_mini_cluster.cc:1428] Setting key 77fd88ac66ef0467cff6330d48d5300d
Aug 09 15:15:41 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288541, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:15:41.993814 28283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:41.967047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:47547 (local address 127.15.11.190:37707)
0809 15:15:41.967554 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:41.967560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:41.971405 (+ 3845us) server_negotiation.cc:407] Connection header received
0809 15:15:41.971482 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:41.971487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:41.971599 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:41.971712 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:15:41.974149 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:41.975310 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:41.977929 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:41.978696 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:41.978840 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:41.985186 (+ 6346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:15:41.985195 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:15:41.985202 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:15:41.985249 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:15:41.989480 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:41.990483 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:41.990491 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:41.990497 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:41.990649 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:41.991260 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:41.991267 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:41.991273 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:41.991721 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:15:41.991953 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:41.993037 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0809 15:15:41.993308 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":244,"threads_started":1}
I20260809 15:15:41.994974 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:41.967555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37707 (local address 127.15.11.130:47547)
0809 15:15:41.968449 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0809 15:15:41.968509 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 15:15:41.970958 (+ 2449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:15:41.972261 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:15:41.972273 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:15:41.973004 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:15:41.973855 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:41.973884 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:41.975510 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:41.975516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:41.977750 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:41.977769 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:41.978977 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:41.979002 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:41.979369 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:41.980279 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:15:41.980321 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:15:41.984887 (+ 4566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:15:41.989649 (+ 4762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:15:41.989661 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:15:41.989681 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:15:41.990322 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:15:41.990779 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:15:41.990836 (+ 57us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:15:41.990842 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:15:41.991120 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:15:41.992033 (+ 913us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:15:41.992042 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:15:41.992530 (+ 488us) client_negotiation.cc:769] Sending connection context
0809 15:15:41.992879 (+ 349us) client_negotiation.cc:240] Negotiation successful
0809 15:15:41.993446 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":315,"threads_started":1}
I20260809 15:15:41.996856 28281 heartbeater.cc:347] Connected to a master server at 127.15.11.190:37707
I20260809 15:15:41.997287 28281 heartbeater.cc:464] Registering TS with master...
I20260809 15:15:41.998376 28281 heartbeater.cc:511] Master 127.15.11.190:37707 requested a full tablet report, sending...
I20260809 15:15:42.000494 27869 ts_manager.cc:194] Registered new tserver with Master: c884ec24c02f4b499a6c0ac20e3d0669 (127.15.11.130:44139)
I20260809 15:15:42.001695 27869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:47547
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:15:42.037349 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:57121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37707
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:15:42.218255 28109 heartbeater.cc:503] Master 127.15.11.190:37707 was elected leader, sending a full tablet report...
W20260809 15:15:42.388643 28287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:15:42.388911 28287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:15:42.389066 28287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:15:42.396646 28287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:15:42.396775 28287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:15:42.396936 28287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:15:42.406704 28287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.11.148:57121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:15:42.408110 28287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:15:42.409521 28287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:15:42.422681 28293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:15:42.422734 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:15:42.423585 28296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:15:42.424049 28287 server_base.cc:1034] running on GCE node
I20260809 15:15:42.424790 28287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:15:42.425711 28287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:15:42.426946 28287 hybrid_clock.cc:648] HybridClock initialized: now 1786288542426918 us; error 49 us; skew 500 ppm
I20260809 15:15:42.427250 28287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:15:42 dist-test-slave-8w9v krb5kdc[27332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288542, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:15:42.430435 28287 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:15:42.431815 28287 webserver.cc:460] Webserver started at http://127.15.11.131:40211/ using document root <none> and password file <none>
I20260809 15:15:42.432559 28287 fs_manager.cc:359] Metadata directory not provided
I20260809 15:15:42.432709 28287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:15:42.433019 28287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:15:42 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288542, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:15:42.448864 28287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb"
format_stamp: "Formatted at 2026-08-09 15:15:42 on dist-test-slave-8w9v"
server_key: "980642d21557ad07ce1056dcf88a46b5"
server_key_iv: "a64fff4bf3df46e3cb88840b9cfc94a3"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:42.449676 28287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb"
format_stamp: "Formatted at 2026-08-09 15:15:42 on dist-test-slave-8w9v"
server_key: "980642d21557ad07ce1056dcf88a46b5"
server_key_iv: "a64fff4bf3df46e3cb88840b9cfc94a3"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:42.455667 28287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:15:42.473417 28304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:42.475026 28287 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 15:15:42.475219 28287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb"
format_stamp: "Formatted at 2026-08-09 15:15:42 on dist-test-slave-8w9v"
server_key: "980642d21557ad07ce1056dcf88a46b5"
server_key_iv: "a64fff4bf3df46e3cb88840b9cfc94a3"
server_key_version: "kuduclusterkey@0"
I20260809 15:15:42.475428 28287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:15:42.509306 28287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:15:42.515120 28287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:15:42.515496 28287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:15:42.516827 28287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:15:42.518993 28287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:15:42.519112 28287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:42.519259 28287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:15:42.519333 28287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:15:42.557045 28287 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:46869
I20260809 15:15:42.557070 28428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:46869 every 8 connection(s)
I20260809 15:15:42.558321 28287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:15:42.566736 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 28287
I20260809 15:15:42.566905 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:15:42.567446 15406 external_mini_cluster.cc:1428] Setting key b22c68f83f7d872de43a7cf6d2a06c9f
Aug 09 15:15:42 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288542, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:15:42.578091 28431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.560863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:34053 (local address 127.15.11.190:37707)
0809 15:15:42.561370 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.561377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.562189 (+ 812us) server_negotiation.cc:407] Connection header received
0809 15:15:42.563860 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.563864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.563943 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.564060 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:15:42.565451 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.566248 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.568432 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.568945 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.569032 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.572677 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:15:42.572682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:15:42.572685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:15:42.572714 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:15:42.575492 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:42.576203 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:42.576207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:42.576209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:42.576329 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:42.576706 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:42.576709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:42.576711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:42.576964 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:15:42.577069 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.577630 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.577803 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":218,"threads_started":1}
I20260809 15:15:42.578890 28432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.561282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37707 (local address 127.15.11.131:34053)
0809 15:15:42.562012 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 15:15:42.562066 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 15:15:42.563625 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:15:42.564323 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:15:42.564331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:15:42.564736 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:15:42.565281 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.565297 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.566405 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.566411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.568281 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.568308 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.569184 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.569192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.569479 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.570230 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:15:42.570257 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:15:42.572480 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:15:42.575623 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:15:42.575629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:15:42.575639 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:15:42.576047 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:15:42.576421 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:15:42.576424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:15:42.576426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:15:42.576609 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:15:42.577064 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:15:42.577069 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:15:42.577323 (+ 254us) client_negotiation.cc:769] Sending connection context
0809 15:15:42.577539 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 15:15:42.577876 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":276,"threads_started":1}
I20260809 15:15:42.580225 28429 heartbeater.cc:347] Connected to a master server at 127.15.11.190:37707
I20260809 15:15:42.580515 28429 heartbeater.cc:464] Registering TS with master...
I20260809 15:15:42.581306 28429 heartbeater.cc:511] Master 127.15.11.190:37707 requested a full tablet report, sending...
I20260809 15:15:42.583079 27868 ts_manager.cc:194] Registered new tserver with Master: d3d0ab67a0fa4291b5b98fb25b5433bb (127.15.11.131:46869)
I20260809 15:15:42.583993 27868 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:34053
I20260809 15:15:42.594973 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:15:42.611392 28431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.599312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.15.11.190:37707)
0809 15:15:42.599528 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.599536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.599936 (+ 400us) server_negotiation.cc:407] Connection header received
0809 15:15:42.600060 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.600063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.600184 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.600261 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:15:42.601679 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.602351 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.603644 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.604037 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.604180 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.606463 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:15:42.606467 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:15:42.606470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:15:42.606499 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:15:42.609099 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:42.609748 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:42.609752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:42.609754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:42.609870 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:42.610192 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:42.610195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:42.610197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:42.610440 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:15:42.610624 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.610901 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.611058 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:15:42.615558 27868 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:15:42.622603 27868 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:15:42.654841 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.634721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:44139 (local address 127.0.0.1:40882)
0809 15:15:42.635562 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 15:15:42.635600 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:15:42.635781 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:15:42.647579 (+ 11798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:15:42.647585 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:15:42.647651 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:15:42.648187 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.648198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.649993 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.650000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.651714 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.651726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.653671 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.653676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.653970 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.654044 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 15:15:42.654203 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 15:15:42.654393 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"spinlock_wait_cycles":13568,"thread_start_us":556,"threads_started":1}
I20260809 15:15:42.655154 28448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.636737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35208 (local address 127.15.11.131:46869)
0809 15:15:42.644912 (+ 8175us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.644925 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.646373 (+ 1448us) server_negotiation.cc:407] Connection header received
0809 15:15:42.646495 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.646503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.646821 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.647075 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:15:42.647892 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.649831 (+ 1939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.652558 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.653997 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.654125 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.654361 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.654580 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.654751 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7938,"thread_start_us":7759,"threads_started":1}
I20260809 15:15:42.655036 28445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.637158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:46869 (local address 127.0.0.1:35208)
0809 15:15:42.645805 (+ 8647us) negotiation.cc:107] Waiting for socket to connect
0809 15:15:42.645825 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:15:42.646031 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:15:42.647110 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:15:42.647120 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:15:42.647150 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:15:42.647671 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.647684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.650042 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.650045 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.652384 (+ 2339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.652399 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.654222 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.654227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.654398 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.654428 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:15:42.654519 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:15:42.654640 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8490,"mutex_wait_us":115,"thread_start_us":7982,"threads_started":1}
I20260809 15:15:42.656199 28444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.634937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40882 (local address 127.15.11.130:44139)
0809 15:15:42.646971 (+ 12034us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.646981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.647010 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:15:42.647121 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.647127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.647350 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.648233 (+ 883us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:15:42.648485 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.649831 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.651889 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.653313 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.655387 (+ 2074us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.655546 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.655625 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.655802 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11861,"thread_start_us":11764,"threads_started":1}
I20260809 15:15:42.656415 28449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.635311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:39097 (local address 127.0.0.1:60614)
0809 15:15:42.645429 (+ 10118us) negotiation.cc:107] Waiting for socket to connect
0809 15:15:42.645455 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:15:42.645630 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:15:42.647441 (+ 1811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:15:42.647453 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:15:42.647485 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:15:42.648106 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.648146 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.650481 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.650486 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.653045 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.653059 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.655239 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.655245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.655760 (+ 515us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.655784 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:15:42.655876 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 15:15:42.656010 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9968,"mutex_wait_us":4,"spinlock_wait_cycles":3328,"thread_start_us":10207,"threads_started":1}
I20260809 15:15:42.656467 28446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.635941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60614 (local address 127.15.11.129:39097)
0809 15:15:42.646717 (+ 10776us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.646726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.646755 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:15:42.646855 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.646861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.647199 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.647414 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:15:42.648632 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.649850 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.653217 (+ 3367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.654695 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.654829 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.655071 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.655947 (+ 876us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.656148 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10571,"thread_start_us":10335,"threads_started":1}
I20260809 15:15:42.659905 28357 tablet_service.cc:1467] Processing CreateTablet for tablet f4b8ce6e1b4a4a0ca4d5146d75694530 (DEFAULT_TABLE table=test-table [id=f6c14d7e66d04e1ea5f4ac46286bca73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:15:42.660283 28201 tablet_service.cc:1467] Processing CreateTablet for tablet f4b8ce6e1b4a4a0ca4d5146d75694530 (DEFAULT_TABLE table=test-table [id=f6c14d7e66d04e1ea5f4ac46286bca73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:15:42.660439 28036 tablet_service.cc:1467] Processing CreateTablet for tablet f4b8ce6e1b4a4a0ca4d5146d75694530 (DEFAULT_TABLE table=test-table [id=f6c14d7e66d04e1ea5f4ac46286bca73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:15:42.661532 28201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4b8ce6e1b4a4a0ca4d5146d75694530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:15:42.661587 28357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4b8ce6e1b4a4a0ca4d5146d75694530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:15:42.661794 28036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4b8ce6e1b4a4a0ca4d5146d75694530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:15:42.673950 28450 tablet_bootstrap.cc:492] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: Bootstrap starting.
I20260809 15:15:42.676947 28451 tablet_bootstrap.cc:492] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: Bootstrap starting.
I20260809 15:15:42.678762 28450 tablet_bootstrap.cc:654] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: Neither blocks nor log segments found. Creating new log.
I20260809 15:15:42.680223 28450 log.cc:824] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: Log is configured to *not* fsync() on all Append() calls
I20260809 15:15:42.680652 28452 tablet_bootstrap.cc:492] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: Bootstrap starting.
I20260809 15:15:42.682227 28451 tablet_bootstrap.cc:654] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:15:42.683415 28450 tablet_bootstrap.cc:492] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: No bootstrap required, opened a new log
I20260809 15:15:42.683604 28451 log.cc:824] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:15:42.683808 28450 ts_tablet_manager.cc:1397] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 15:15:42.685035 28452 tablet_bootstrap.cc:654] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: Neither blocks nor log segments found. Creating new log.
I20260809 15:15:42.686578 28452 log.cc:824] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: Log is configured to *not* fsync() on all Append() calls
I20260809 15:15:42.690459 28452 tablet_bootstrap.cc:492] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: No bootstrap required, opened a new log
I20260809 15:15:42.690779 28452 ts_tablet_manager.cc:1397] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260809 15:15:42.692391 28450 raft_consensus.cc:348] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } }
I20260809 15:15:42.692759 28450 raft_consensus.cc:374] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:15:42.692871 28450 raft_consensus.cc:729] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c884ec24c02f4b499a6c0ac20e3d0669, State: Initialized, Role: FOLLOWER
I20260809 15:15:42.692925 28451 tablet_bootstrap.cc:492] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: No bootstrap required, opened a new log
I20260809 15:15:42.693178 28451 ts_tablet_manager.cc:1397] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20260809 15:15:42.693478 28450 consensus_queue.cc:260] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } }
I20260809 15:15:42.695304 28450 ts_tablet_manager.cc:1428] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260809 15:15:42.695819 28281 heartbeater.cc:503] Master 127.15.11.190:37707 was elected leader, sending a full tablet report...
I20260809 15:15:42.700194 28452 raft_consensus.cc:348] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } }
I20260809 15:15:42.700675 28452 raft_consensus.cc:374] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:15:42.700822 28452 raft_consensus.cc:729] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3d0ab67a0fa4291b5b98fb25b5433bb, State: Initialized, Role: FOLLOWER
I20260809 15:15:42.701542 28452 consensus_queue.cc:260] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } }
I20260809 15:15:42.702733 28429 heartbeater.cc:503] Master 127.15.11.190:37707 was elected leader, sending a full tablet report...
I20260809 15:15:42.703269 28452 ts_tablet_manager.cc:1428] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260809 15:15:42.703685 28451 raft_consensus.cc:348] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } }
I20260809 15:15:42.704212 28451 raft_consensus.cc:374] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:15:42.704326 28451 raft_consensus.cc:729] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9a9c07b9e8403e92ba0fc9204ff7f4, State: Initialized, Role: FOLLOWER
I20260809 15:15:42.704870 28451 consensus_queue.cc:260] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } }
I20260809 15:15:42.706590 28451 ts_tablet_manager.cc:1428] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260809 15:15:42.713160 28460 raft_consensus.cc:482] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:15:42.713356 28460 raft_consensus.cc:504] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } }
I20260809 15:15:42.714815 28460 leader_election.cc:288] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c884ec24c02f4b499a6c0ac20e3d0669 (127.15.11.130:44139), d3d0ab67a0fa4291b5b98fb25b5433bb (127.15.11.131:46869)
W20260809 15:15:42.716650 28282 tablet.cc:2369] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:15:42.721607 28444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.715088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:37437 (local address 127.15.11.130:44139)
0809 15:15:42.715269 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.715274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.715841 (+ 567us) server_negotiation.cc:407] Connection header received
0809 15:15:42.716063 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.716070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.716192 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.716288 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:15:42.717048 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.717959 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.719801 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.720680 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.720852 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.720941 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.721205 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.721329 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:15:42.721758 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.715181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:44139 (local address 127.15.11.129:37437)
0809 15:15:42.715732 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 15:15:42.715748 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:15:42.715911 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:15:42.716352 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:15:42.716364 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:15:42.716396 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:15:42.716827 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.716843 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.718096 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.718103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.719654 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.719666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.720877 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.720884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.721044 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.721067 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:15:42.721179 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 15:15:42.721302 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":48,"thread_start_us":311,"threads_started":1}
I20260809 15:15:42.721819 28448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.715964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:51269 (local address 127.15.11.131:46869)
0809 15:15:42.716242 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.716248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.716638 (+ 390us) server_negotiation.cc:407] Connection header received
0809 15:15:42.716827 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.716835 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.716915 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.717040 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:15:42.717571 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.718259 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.719998 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.720886 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.721065 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.721138 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.721446 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.721573 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:15:42.721969 28462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.715858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:46869 (local address 127.15.11.129:51269)
0809 15:15:42.716526 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0809 15:15:42.716540 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:15:42.716691 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:15:42.717060 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:15:42.717064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:15:42.717089 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:15:42.717424 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.717433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.718409 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.718418 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.719837 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:15:42.719849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.721102 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.721108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:15:42.721253 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.721276 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:15:42.721419 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 15:15:42.721545 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":4,"thread_start_us":350,"threads_started":1}
I20260809 15:15:42.722796 28226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4b8ce6e1b4a4a0ca4d5146d75694530" candidate_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" is_pre_election: true
I20260809 15:15:42.722862 28380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4b8ce6e1b4a4a0ca4d5146d75694530" candidate_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" is_pre_election: true
I20260809 15:15:42.723344 28226 raft_consensus.cc:2454] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9a9c07b9e8403e92ba0fc9204ff7f4 in term 0.
I20260809 15:15:42.723347 28380 raft_consensus.cc:2454] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9a9c07b9e8403e92ba0fc9204ff7f4 in term 0.
I20260809 15:15:42.724082 27988 leader_election.cc:302] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9a9c07b9e8403e92ba0fc9204ff7f4, d3d0ab67a0fa4291b5b98fb25b5433bb; no voters:
I20260809 15:15:42.724543 28460 raft_consensus.cc:2781] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:15:42.724663 28460 raft_consensus.cc:482] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:15:42.724747 28460 raft_consensus.cc:3037] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:15:42.726949 28460 raft_consensus.cc:504] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } }
I20260809 15:15:42.727509 28460 leader_election.cc:288] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [CANDIDATE]: Term 1 election: Requested vote from peers c884ec24c02f4b499a6c0ac20e3d0669 (127.15.11.130:44139), d3d0ab67a0fa4291b5b98fb25b5433bb (127.15.11.131:46869)
I20260809 15:15:42.727921 28226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4b8ce6e1b4a4a0ca4d5146d75694530" candidate_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c884ec24c02f4b499a6c0ac20e3d0669"
I20260809 15:15:42.728009 28380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4b8ce6e1b4a4a0ca4d5146d75694530" candidate_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb"
I20260809 15:15:42.728111 28226 raft_consensus.cc:3037] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:15:42.728274 28380 raft_consensus.cc:3037] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:15:42.730370 28226 raft_consensus.cc:2454] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9a9c07b9e8403e92ba0fc9204ff7f4 in term 1.
I20260809 15:15:42.730441 28380 raft_consensus.cc:2454] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9a9c07b9e8403e92ba0fc9204ff7f4 in term 1.
I20260809 15:15:42.730718 27988 leader_election.cc:302] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9a9c07b9e8403e92ba0fc9204ff7f4, c884ec24c02f4b499a6c0ac20e3d0669; no voters:
I20260809 15:15:42.731019 28460 raft_consensus.cc:2781] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:15:42.731479 28460 raft_consensus.cc:686] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 1 LEADER]: Becoming Leader. State: Replica: 3e9a9c07b9e8403e92ba0fc9204ff7f4, State: Running, Role: LEADER
I20260809 15:15:42.731945 28460 consensus_queue.cc:237] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } }
I20260809 15:15:42.736742 27868 catalog_manager.cc:5495] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e9a9c07b9e8403e92ba0fc9204ff7f4 (127.15.11.129). New cstate: current_term: 1 leader_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9a9c07b9e8403e92ba0fc9204ff7f4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 39097 } health_report { overall_health: HEALTHY } } }
I20260809 15:15:42.755590 28446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.750091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60622 (local address 127.15.11.129:39097)
0809 15:15:42.750337 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.750341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.750359 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:15:42.750539 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.750542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.750626 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.750745 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:15:42.751099 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.751826 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.753396 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.753925 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.754070 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.754489 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:15:42.754991 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:15:42.755101 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.755197 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.755262 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:15:42.763692 28226 raft_consensus.cc:1261] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [term 1 FOLLOWER]: Refusing update from remote peer 3e9a9c07b9e8403e92ba0fc9204ff7f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:15:42.763819 28380 raft_consensus.cc:1261] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [term 1 FOLLOWER]: Refusing update from remote peer 3e9a9c07b9e8403e92ba0fc9204ff7f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:15:42.764778 28463 consensus_queue.cc:1035] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c884ec24c02f4b499a6c0ac20e3d0669" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:15:42.765209 28460 consensus_queue.cc:1035] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3d0ab67a0fa4291b5b98fb25b5433bb" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 46869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:15:42.794322 27868 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:15:42.794608 27868 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:15:42.798347 27868 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc47db25939a43f3bcd594e247886c7f: Sending DeleteTablet for 3 replicas of tablet f4b8ce6e1b4a4a0ca4d5146d75694530
I20260809 15:15:42.799718 28201 tablet_service.cc:1514] Processing DeleteTablet for tablet f4b8ce6e1b4a4a0ca4d5146d75694530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:15:42 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:40882
I20260809 15:15:42.799758 28357 tablet_service.cc:1514] Processing DeleteTablet for tablet f4b8ce6e1b4a4a0ca4d5146d75694530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:15:42 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:35208
I20260809 15:15:42.800581 28036 tablet_service.cc:1514] Processing DeleteTablet for tablet f4b8ce6e1b4a4a0ca4d5146d75694530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:15:42 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:60614
I20260809 15:15:42.801541 28478 tablet_replica.cc:330] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: stopping tablet replica
I20260809 15:15:42.802006 28478 raft_consensus.cc:2229] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:15:42.802695 28478 raft_consensus.cc:2258] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:15:42.802754 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 27962
I20260809 15:15:42.813717 28478 ts_tablet_manager.cc:1905] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:15:42.816329 28476 tablet_replica.cc:330] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: stopping tablet replica
I20260809 15:15:42.816958 28476 raft_consensus.cc:2229] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:15:42.817569 28476 raft_consensus.cc:2258] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:15:42.819950 28476 ts_tablet_manager.cc:1905] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 15:15:42 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:15:42.825409 28477 tablet_replica.cc:330] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: stopping tablet replica
I20260809 15:15:42.825877 28477 raft_consensus.cc:2229] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:15:42.826617 28477 raft_consensus.cc:2258] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:15:42.827126 28476 ts_tablet_manager.cc:1918] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:15:42.827333 28476 log.cc:1196] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/f4b8ce6e1b4a4a0ca4d5146d75694530
I20260809 15:15:42.827978 28476 ts_tablet_manager.cc:1939] T f4b8ce6e1b4a4a0ca4d5146d75694530 P c884ec24c02f4b499a6c0ac20e3d0669: Deleting consensus metadata
I20260809 15:15:42.829903 28478 ts_tablet_manager.cc:1918] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:15:42.830082 28478 log.cc:1196] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/f4b8ce6e1b4a4a0ca4d5146d75694530
I20260809 15:15:42.830077 27855 catalog_manager.cc:4841] TS c884ec24c02f4b499a6c0ac20e3d0669 (127.15.11.130:44139): tablet f4b8ce6e1b4a4a0ca4d5146d75694530 (table test-table [id=f6c14d7e66d04e1ea5f4ac46286bca73]) successfully deleted
I20260809 15:15:42.830693 28478 ts_tablet_manager.cc:1939] T f4b8ce6e1b4a4a0ca4d5146d75694530 P 3e9a9c07b9e8403e92ba0fc9204ff7f4: Deleting consensus metadata
I20260809 15:15:42.833232 27854 catalog_manager.cc:4841] TS 3e9a9c07b9e8403e92ba0fc9204ff7f4 (127.15.11.129:39097): tablet f4b8ce6e1b4a4a0ca4d5146d75694530 (table test-table [id=f6c14d7e66d04e1ea5f4ac46286bca73]) successfully deleted
I20260809 15:15:42.833600 28477 ts_tablet_manager.cc:1905] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:15:42.835246 28446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.803892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60636 (local address 127.15.11.129:39097)
0809 15:15:42.804092 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.804099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.812663 (+ 8564us) server_negotiation.cc:407] Connection header received
0809 15:15:42.812851 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.812856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.812977 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.813086 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:15:42.815406 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.816678 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.819697 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.820653 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.820760 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.826777 (+ 6017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:15:42.826791 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:15:42.826809 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:15:42.826882 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:15:42.831658 (+ 4776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:42.832654 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:42.832663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:42.832666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:42.832853 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:42.833377 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:42.833383 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:42.833386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:42.833842 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:15:42.834171 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.834512 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.834758 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:15:42.835938 28104 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:15:42.840895 28477 ts_tablet_manager.cc:1918] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:15:42.841075 28477 log.cc:1196] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/f4b8ce6e1b4a4a0ca4d5146d75694530
I20260809 15:15:42.841516 28477 ts_tablet_manager.cc:1939] T f4b8ce6e1b4a4a0ca4d5146d75694530 P d3d0ab67a0fa4291b5b98fb25b5433bb: Deleting consensus metadata
I20260809 15:15:42.842993 27855 catalog_manager.cc:4841] TS d3d0ab67a0fa4291b5b98fb25b5433bb (127.15.11.131:46869): tablet f4b8ce6e1b4a4a0ca4d5146d75694530 (table test-table [id=f6c14d7e66d04e1ea5f4ac46286bca73]) successfully deleted
I20260809 15:15:42.936390 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 27962
I20260809 15:15:42.958039 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 28115
Aug 09 15:15:42 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:15:42.976507 28444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:42.958823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.15.11.130:44139)
0809 15:15:42.959000 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:42.959005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:42.959022 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:15:42.959159 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:42.959162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:42.959226 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:42.959320 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:15:42.960696 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.961592 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.963509 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:42.963941 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:42.964049 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:42.969267 (+ 5218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:15:42.969280 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:15:42.969297 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:15:42.969337 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:15:42.973554 (+ 4217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:42.974239 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:42.974251 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:42.974256 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:42.974441 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:42.974787 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:42.974792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:42.974795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:42.975269 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:15:42.975586 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:42.975751 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 15:15:42.975965 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:15:42.977228 28273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:15:43.068276 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 28115
I20260809 15:15:43.088712 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 28287
Aug 09 15:15:43 dist-test-slave-8w9v krb5kdc[27332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:15:43.103082 28448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:15:43.089455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35212 (local address 127.15.11.131:46869)
0809 15:15:43.089618 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 15:15:43.089622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:15:43.089640 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:15:43.089804 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:15:43.089808 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:15:43.089881 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:15:43.089960 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:15:43.091257 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:43.091968 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:43.093264 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:15:43.093622 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:15:43.093730 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:15:43.097916 (+ 4186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:15:43.097931 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:15:43.097949 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:15:43.097981 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:15:43.100853 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:43.101414 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:43.101422 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:43.101424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:43.101548 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:15:43.101844 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:15:43.101847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:15:43.101849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:15:43.102198 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:15:43.102440 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 15:15:43.102649 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 15:15:43.102820 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:15:43.103739 28423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:15:43.193645 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 28287
I20260809 15:15:43.214311 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 27833
I20260809 15:15:43.215551 27903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:15:43.327091 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 27833
I20260809 15:15:43.353670 15406 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 15:15:43.699790 15406 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 15:15:43.700026 15406 mini_ranger.cc:66] Stopping Ranger...
I20260809 15:15:44.044516 15406 mini_ranger.cc:68] Stopped Ranger
2026-08-09 15:15:44.044 UTC [27366] LOG: received smart shutdown request
2026-08-09 15:15:44.048 UTC [27366] LOG: background worker "logical replication launcher" (PID 27376) exited with exit code 1
2026-08-09 15:15:44.048 UTC [27370] LOG: shutting down
2026-08-09 15:15:44.207 UTC [27366] LOG: database system is shut down
2026-08-09T15:15:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50640 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:15:44 dist-test-slave-8w9v krb5kdc[28492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:15:44 dist-test-slave-8w9v krb5kdc[28492](info): set up 2 sockets
Aug 09 15:15:44 dist-test-slave-8w9v krb5kdc[28492](info): commencing operation
krb5kdc: starting...
W20260809 15:15:45.493584 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:15:45 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:15:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:15:45Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/rangeradmin_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/rangeradmin_127.15.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/rangerlookup_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/rangerlookup_127.15.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/HTTP_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/HTTP_127.15.11.148@KRBTEST.COM.keytab.
I20260809 15:15:45.847065 15406 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskFizLz9/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/postgres -l logfile start
2026-08-09 15:15:48.183 UTC [28528] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 15:15:48.183 UTC [28528] LOG: listening on IPv4 address "127.15.11.148", port 53765
2026-08-09 15:15:48.186 UTC [28528] LOG: listening on Unix socket "/tmp/.s.PGSQL.53765"
2026-08-09 15:15:48.208 UTC [28532] LOG: database system was shut down at 2026-08-09 15:15:46 UTC
2026-08-09 15:15:48.211 UTC [28528] LOG: database system is ready to accept connections
I20260809 15:15:49.211846 15406 mini_postgres.cc:96] Postgres bound to 53765
2026-08-09 15:15:49.234 UTC [28540] FATAL: database "slave" does not exist
127.15.11.148:53765 - accepting connections
I20260809 15:15:49.234886 15406 mini_ranger.cc:161] Starting Ranger...
I20260809 15:15:49.263608 15406 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 15:15:49.703935 15406 mini_ranger.cc:87] Created ranger Postgres database
I20260809 15:15:49.704069 15406 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-admin
2026-08-09 15:15:49,988 [I] DB FLAVOR :POSTGRES
2026-08-09 15:15:49,988 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:15:49,989 [I] Checking connection..
2026-08-09 15:15:49,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 15:15:50,228 [I] Checking connection passed.
2026-08-09 15:15:50,228 [I] --------- Verifying version history table ---------
2026-08-09 15:15:50,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:15:50,486 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:15:50,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:15:50,730 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:15:50,730 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 15:15:50,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 15:15:50.984 UTC [28610] WARNING: there is no transaction in progress
2026-08-09 15:15:50,991 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 15:15:50,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:15:51,246 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 15:15:51,247 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:15:51,247 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 15:15:51,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 15:15:51,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8w9v.c.gcp-upstream.internal','N') ;"
2026-08-09 15:15:52,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:15:52,255 [I] Table x_portal_user does not exist in database ranger
2026-08-09 15:15:52,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:15:52,513 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 15:15:52,513 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:15:52,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:15:53,027 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 15:15:53,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 15:15:53.957 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:53.973 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:53.989 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.004 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.023 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.041 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.101 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.108 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.122 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.131 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.141 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.154 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.164 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.170 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.177 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54.183 UTC [28728] WARNING: there is no transaction in progress
2026-08-09 15:15:54,767 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 15:15:54,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 15:15:55,013 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 15:15:55,013 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:15:55,274 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 15:15:55,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:15:55,536 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 15:15:55,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:15:55,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:15:56,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8w9v.c.gcp-upstream.internal';"
2026-08-09 15:15:56,274 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 15:15:56,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 15:15:56,532 [I] DB_PATCHES have already been applied
I20260809 15:15:56.538240 15406 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-admin:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskFizLz9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 15:15:56.538373 15406 mini_ranger.cc:193] Using host: 127.15.11.148
I20260809 15:15:56.558168 15406 mini_ranger.cc:227] Ranger admin URL: http://127.15.11.148:59091
Aug 09, 2026 3:15:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:15:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:15:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:15:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:15:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:15:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:15:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 15:15:57.732949 15406 mini_ranger.cc:160] Time spent starting Ranger: real 8.498s user 0.007s sys 0.046s
Aug 09, 2026 3:16:17 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:16:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 3:16:17 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:16:17 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:16:18 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 15:16:18.981683 15406 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/rangerkms_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/rangerkms_127.15.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.11.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.11.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.11.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/HTTP_127.15.11.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.11.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/HTTP_127.15.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 15:16:19 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288579, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 15:16:19.169665 15406 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 15:16:19.192528 15406 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 15:16:19.984949 15406 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 15:16:19.985077 15406 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-kms
I20260809 15:16:19.985126 15406 mini_ranger_kms.cc:227] Using postgres at 127.15.11.148:53765
2026-08-09 15:16:20,382 [I] DB FLAVOR :POSTGRES
2026-08-09 15:16:20,382 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:16:20,382 [I] Checking connection
2026-08-09 15:16:20,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 15:16:20,611 [I] connection success
2026-08-09 15:16:20,612 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 15:16:20,612 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 15:16:20,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 15:16:20,866 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 15:16:20,866 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:16:20,866 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 15:16:20,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.11.148:53765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 15:16:21,123 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 15:16:21.572990 15406 mini_ranger_kms.cc:310] Created kms service
I20260809 15:16:21.743450 15406 mini_ranger_kms.cc:326] Created kudu user
I20260809 15:16:21.795181 15406 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 15:16:21 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288579, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:22.242457 15406 mini_ranger_kms.cc:384] Added ranger policy
I20260809 15:16:22.242831 15406 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-kms:/tmp/dist-test-taskFizLz9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskFizLz9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 15:16:22.242933 15406 mini_ranger_kms.cc:240] Using host: 127.15.11.148
I20260809 15:16:22.263852 15406 mini_ranger_kms.cc:276] Ranger KMS PID: 28941
I20260809 15:16:22.264024 15406 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.11.148:44861
Aug 09, 2026 3:16:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:16:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:16:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:16:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:16:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:16:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 15:16:24 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288584, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.11.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:24.660588 15406 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.491s user 0.055s sys 0.370s
I20260809 15:16:24.661007 15406 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 15:16:24.661233 15406 mini_ranger_kms.cc:401] 127.15.11.148:44861/kms/v1/keys
Aug 09 15:16:24 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288584, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.11.148@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
Aug 09 15:16:25 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288584, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.11.148@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
Aug 09 15:16:27 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:16:27.609853 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:42109
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:42109
--ranger_config_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:44861/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:27.957129 28993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:27.957412 28993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:27.957576 28993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:27.964659 28993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:16:27.964775 28993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:27.964859 28993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:27.964952 28993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:16:27.965037 28993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:16:27.974571 28993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.11.148:44861/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:42109
--ranger_config_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:42109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:27.975781 28993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:27.977147 28993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:27.989413 28998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:27.989452 29002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:27.989494 28999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:27.990139 28993 server_base.cc:1034] running on GCE node
I20260809 15:16:27.990963 28993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:27.992415 28993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:27.993642 28993 hybrid_clock.cc:648] HybridClock initialized: now 1786288587993600 us; error 52 us; skew 500 ppm
I20260809 15:16:27.993997 28993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:27 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288587, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:28.000577 28993 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:16:28.002008 28993 webserver.cc:460] Webserver started at http://127.15.11.190:42233/ using document root <none> and password file <none>
I20260809 15:16:28.002606 28993 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:28.002704 28993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:28.002980 28993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:16:28 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288587, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:28.061262 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "f0eec95e269d4fbe8e8365731a5dff6b"
format_stamp: "Formatted at 2026-08-09 15:16:28 on dist-test-slave-8w9v"
server_key: "659c82b913b6e401383ad4bc1163f397"
server_key_iv: "df50a1ef58d3e05401f4ed8a9effe915"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:28.062135 28993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "f0eec95e269d4fbe8e8365731a5dff6b"
format_stamp: "Formatted at 2026-08-09 15:16:28 on dist-test-slave-8w9v"
server_key: "659c82b913b6e401383ad4bc1163f397"
server_key_iv: "df50a1ef58d3e05401f4ed8a9effe915"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:28.068248 28993 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260809 15:16:28.084712 29011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:28.086511 28993 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 15:16:28.086709 28993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "f0eec95e269d4fbe8e8365731a5dff6b"
format_stamp: "Formatted at 2026-08-09 15:16:28 on dist-test-slave-8w9v"
server_key: "659c82b913b6e401383ad4bc1163f397"
server_key_iv: "df50a1ef58d3e05401f4ed8a9effe915"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:28.086946 28993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:28.110678 28993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:28.114933 28993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:28.115276 28993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:28.134773 28993 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:42109
I20260809 15:16:28.134806 29072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:42109 every 8 connection(s)
I20260809 15:16:28.136258 28993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:16:28.139984 29073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:28.140004 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 28993
I20260809 15:16:28.140275 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:16:28.141027 15406 external_mini_cluster.cc:1428] Setting key 4fb6a893399cce2b1210fe963b49d9bd
I20260809 15:16:28.153209 29073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Bootstrap starting.
Aug 09 15:16:28 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:28.156991 29073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:28.158432 29073 log.cc:824] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:28.161724 29073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: No bootstrap required, opened a new log
I20260809 15:16:28.164844 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:28.143063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59784 (local address 127.15.11.190:42109)
0809 15:16:28.143902 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:28.143916 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:28.143958 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:16:28.145915 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:28.145937 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:28.146320 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:28.146612 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:28.148265 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:28.149535 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:28.151393 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:28.152014 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:28.152202 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:28.156690 (+ 4488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:28.156704 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:28.156722 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:28.156802 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:28.160642 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:28.161277 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:28.161285 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:28.161292 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:28.161510 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:28.161893 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:28.161901 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:28.161903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:28.162403 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:28.162736 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:28.163147 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 15:16:28.163597 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":379,"threads_started":1}
I20260809 15:16:28.169905 29073 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } }
I20260809 15:16:28.170256 29073 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:28.170360 29073 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0eec95e269d4fbe8e8365731a5dff6b, State: Initialized, Role: FOLLOWER
I20260809 15:16:28.170867 29073 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } }
I20260809 15:16:28.171105 29073 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:16:28.171250 29073 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:16:28.171409 29073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:28.173534 29073 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } }
I20260809 15:16:28.174024 29073 leader_election.cc:302] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0eec95e269d4fbe8e8365731a5dff6b; no voters:
I20260809 15:16:28.174582 29073 leader_election.cc:288] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:16:28.174721 29079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:16:28.175508 29079 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 1 LEADER]: Becoming Leader. State: Replica: f0eec95e269d4fbe8e8365731a5dff6b, State: Running, Role: LEADER
I20260809 15:16:28.176172 29079 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } }
I20260809 15:16:28.177032 29073 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:16:28.179272 29080 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } } }
I20260809 15:16:28.179633 29080 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:28.180501 29081 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0eec95e269d4fbe8e8365731a5dff6b. Latest consensus state: current_term: 1 leader_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } } }
I20260809 15:16:28.180796 29081 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:28.182468 29087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:16:28.187728 29087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:16:28.188431 29073 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 15:16:28.206208 29087 catalog_manager.cc:1347] Generated new cluster ID: 9feb0abcbba84441ac407b2024c97764
I20260809 15:16:28.206533 29087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:16:28.227591 29087 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:16:28.229727 29087 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:16:28.255412 29087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Generated new TSK 0
I20260809 15:16:28.256620 29087 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:16:29.003736 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:44861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42109
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:16:29 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288589, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 15:16:29.434583 29123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:29.434998 29123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:29.435209 29123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:29.446949 29123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:29.447131 29123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:29.447337 29123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:16:29.463332 29123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.11.148:44861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:29.465085 29123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:29.466919 29123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:29.484640 29134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:29.485257 29133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:29.490793 29123 server_base.cc:1034] running on GCE node
W20260809 15:16:29.491482 29136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:29.492453 29123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:29.493852 29123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:29.495153 29123 hybrid_clock.cc:648] HybridClock initialized: now 1786288589495085 us; error 56 us; skew 500 ppm
I20260809 15:16:29.495597 29123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:29 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288589, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:29.499996 29123 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:16:29.507632 29123 webserver.cc:460] Webserver started at http://127.15.11.129:33871/ using document root <none> and password file <none>
I20260809 15:16:29.508635 29123 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:29.508785 29123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:29.509105 29123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:16:29 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288589, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:29.534260 29123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "0d6f183adff6439aa158fc2b962c2c48"
format_stamp: "Formatted at 2026-08-09 15:16:29 on dist-test-slave-8w9v"
server_key: "c2151887643ef5c7ded94d55d1e13702"
server_key_iv: "5574cfabdbd25d5850fb4dc82199d3fd"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:29.535176 29123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "0d6f183adff6439aa158fc2b962c2c48"
format_stamp: "Formatted at 2026-08-09 15:16:29 on dist-test-slave-8w9v"
server_key: "c2151887643ef5c7ded94d55d1e13702"
server_key_iv: "5574cfabdbd25d5850fb4dc82199d3fd"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:29.540997 29123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:16:29.561679 29144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:29.575824 29123 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.004s sys 0.002s
I20260809 15:16:29.576175 29123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "0d6f183adff6439aa158fc2b962c2c48"
format_stamp: "Formatted at 2026-08-09 15:16:29 on dist-test-slave-8w9v"
server_key: "c2151887643ef5c7ded94d55d1e13702"
server_key_iv: "5574cfabdbd25d5850fb4dc82199d3fd"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:29.576550 29123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:29.601317 29123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:29.605711 29123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:29.606114 29123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:29.607194 29123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:29.608732 29123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:29.608891 29123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 15:16:29.609097 29123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:29.609225 29123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:29.696178 29123 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:36569
I20260809 15:16:29.696209 29269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:36569 every 8 connection(s)
I20260809 15:16:29.697738 29123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:16:29.708110 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 29123
I20260809 15:16:29.708369 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:16:29.709254 15406 external_mini_cluster.cc:1428] Setting key e83f32ad4e14dfedf4f3677ffbcb1d28
Aug 09 15:16:29 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288589, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:29.731071 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:29.705803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:59001 (local address 127.15.11.190:42109)
0809 15:16:29.708002 (+ 2199us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:29.708010 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:29.711505 (+ 3495us) server_negotiation.cc:407] Connection header received
0809 15:16:29.711566 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:29.711571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:29.711662 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:29.711760 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:29.713906 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:29.715140 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:29.717668 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:29.718374 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:29.718499 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:29.723766 (+ 5267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:29.723774 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:29.723783 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:29.723840 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:29.727352 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:29.728233 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:29.728238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:29.728241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:29.728409 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:29.728986 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:29.728991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:29.728995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:29.729332 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:29.729482 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:29.730468 (+ 986us) server_negotiation.cc:299] Negotiation successful
0809 15:16:29.730691 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2047,"thread_start_us":4179,"threads_started":1}
I20260809 15:16:29.733609 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:29.706324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42109 (local address 127.15.11.129:59001)
0809 15:16:29.708615 (+ 2291us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:29.708670 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:29.711124 (+ 2454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:29.712200 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:29.712213 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:29.712854 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:29.713700 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:29.713730 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:29.715364 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:29.715371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:29.717501 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:29.717520 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:29.718860 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:29.718867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:29.719165 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:29.720004 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:29.720045 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:29.723528 (+ 3483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:29.727524 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:29.727532 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:29.727548 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:29.728062 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:29.728590 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:29.728595 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:29.728598 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:29.728851 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:29.729578 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:29.729586 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:29.729998 (+ 412us) client_negotiation.cc:769] Sending connection context
0809 15:16:29.731647 (+ 1649us) client_negotiation.cc:240] Negotiation successful
0809 15:16:29.732198 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1948,"thread_start_us":3619,"threads_started":1}
I20260809 15:16:29.735922 29270 heartbeater.cc:347] Connected to a master server at 127.15.11.190:42109
I20260809 15:16:29.736339 29270 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:29.737351 29270 heartbeater.cc:511] Master 127.15.11.190:42109 requested a full tablet report, sending...
I20260809 15:16:29.739880 29034 ts_manager.cc:194] Registered new tserver with Master: 0d6f183adff6439aa158fc2b962c2c48 (127.15.11.129:36569)
I20260809 15:16:29.741963 29034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:59001
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Aug 09 15:16:29 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288589, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:16:29.801100 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:44861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42109
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:16:30 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288589, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
W20260809 15:16:30.246183 29277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:30.246703 29277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:30.246925 29277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:30.258657 29277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:30.258882 29277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:30.259114 29277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:16:30.274677 29277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.11.148:44861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:30.276678 29277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:30.278550 29277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:30.320425 29286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:30.322980 29284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:30.325676 29290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:30.326177 29277 server_base.cc:1034] running on GCE node
I20260809 15:16:30.327247 29277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:30.333609 29277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:30.334892 29277 hybrid_clock.cc:648] HybridClock initialized: now 1786288590334784 us; error 99 us; skew 500 ppm
I20260809 15:16:30.335340 29277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:30 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288590, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:30.339865 29277 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:16:30.360437 29277 webserver.cc:460] Webserver started at http://127.15.11.130:37949/ using document root <none> and password file <none>
I20260809 15:16:30.361596 29277 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:30.361829 29277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:30.362267 29277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:16:30 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288590, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:30.393435 29277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "d5cb73066ba248ac9d057f740b862b9f"
format_stamp: "Formatted at 2026-08-09 15:16:30 on dist-test-slave-8w9v"
server_key: "e6465b970a28c3154d58d434470fc00e"
server_key_iv: "01aa2e626358cd602dcfba6ed6db104b"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:30.394986 29277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "d5cb73066ba248ac9d057f740b862b9f"
format_stamp: "Formatted at 2026-08-09 15:16:30 on dist-test-slave-8w9v"
server_key: "e6465b970a28c3154d58d434470fc00e"
server_key_iv: "01aa2e626358cd602dcfba6ed6db104b"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:30.408936 29277 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.008s sys 0.004s
I20260809 15:16:30.459148 29298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:30.461226 29277 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.004s sys 0.003s
I20260809 15:16:30.461514 29277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "d5cb73066ba248ac9d057f740b862b9f"
format_stamp: "Formatted at 2026-08-09 15:16:30 on dist-test-slave-8w9v"
server_key: "e6465b970a28c3154d58d434470fc00e"
server_key_iv: "01aa2e626358cd602dcfba6ed6db104b"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:30.461879 29277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:30.525852 29277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:30.531637 29277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:30.532051 29277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:30.539438 29277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:30.547576 29277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:30.547742 29277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 15:16:30.547930 29277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:30.548048 29277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:30.680439 29277 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:43153
I20260809 15:16:30.680464 29428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:43153 every 8 connection(s)
I20260809 15:16:30.682394 29277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:16:30.688913 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 29277
I20260809 15:16:30.689078 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:16:30.689663 15406 external_mini_cluster.cc:1428] Setting key cc6c71bd2002e93f6772fe1e6d25ea24
Aug 09 15:16:30 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288590, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:30.734457 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:30.686507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:39231 (local address 127.15.11.190:42109)
0809 15:16:30.711576 (+ 25069us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:30.711582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:30.712825 (+ 1243us) server_negotiation.cc:407] Connection header received
0809 15:16:30.716277 (+ 3452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:30.716283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:30.716368 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:30.716447 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:30.718703 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:30.719692 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:30.722223 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:30.722822 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:30.722961 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:30.727192 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:30.727199 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:30.727203 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:30.727244 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:30.730781 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:30.731674 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:30.731679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:30.731683 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:30.731838 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:30.732402 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:30.732407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:30.732411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:30.732752 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:30.732895 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:30.733840 (+ 945us) server_negotiation.cc:299] Negotiation successful
0809 15:16:30.734074 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24942,"thread_start_us":25075,"threads_started":1}
I20260809 15:16:30.736694 29431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:30.685939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42109 (local address 127.15.11.130:39231)
0809 15:16:30.712251 (+ 26312us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:30.712329 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:30.714923 (+ 2594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:30.717004 (+ 2081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:30.717027 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:30.717643 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:30.718508 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:30.718532 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:30.719906 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:30.719910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:30.722073 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:30.722089 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:30.723111 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:30.723119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:30.723410 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:30.724259 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:30.724297 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:30.726902 (+ 2605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:30.730979 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:30.730988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:30.731004 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:30.731534 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:30.731996 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:30.732000 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:30.732003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:30.732263 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:30.732988 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:30.732995 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:30.733393 (+ 398us) client_negotiation.cc:769] Sending connection context
0809 15:16:30.734820 (+ 1427us) client_negotiation.cc:240] Negotiation successful
0809 15:16:30.735321 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25969,"thread_start_us":29226,"threads_started":1}
I20260809 15:16:30.738547 29429 heartbeater.cc:347] Connected to a master server at 127.15.11.190:42109
I20260809 15:16:30.739037 29429 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:30.739916 29429 heartbeater.cc:511] Master 127.15.11.190:42109 requested a full tablet report, sending...
I20260809 15:16:30.741554 29034 ts_manager.cc:194] Registered new tserver with Master: d5cb73066ba248ac9d057f740b862b9f (127.15.11.130:43153)
I20260809 15:16:30.742579 29034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:39231
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
I20260809 15:16:30.745630 29270 heartbeater.cc:503] Master 127.15.11.190:42109 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:16:30.788877 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:44861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42109
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:31.176247 29435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:31.176512 29435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:31.176729 29435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:31.184356 29435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:31.184471 29435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:31.184643 29435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:16:31.194382 29435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.11.148:44861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:31.195614 29435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:31.197016 29435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:31.210090 29440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:31.210139 29442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:31.210171 29444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:31.211088 29435 server_base.cc:1034] running on GCE node
I20260809 15:16:31.211654 29435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:31.212613 29435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:31.213809 29435 hybrid_clock.cc:648] HybridClock initialized: now 1786288591213765 us; error 40 us; skew 500 ppm
I20260809 15:16:31.214111 29435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:31 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288591, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:31.217200 29435 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:16:31.218525 29435 webserver.cc:460] Webserver started at http://127.15.11.131:36119/ using document root <none> and password file <none>
I20260809 15:16:31.219152 29435 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:31.219249 29435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:31.219509 29435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:16:31 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288591, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:31.237093 29435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "24f3f4d0572e4169855cf98f9c4fc4ce"
format_stamp: "Formatted at 2026-08-09 15:16:31 on dist-test-slave-8w9v"
server_key: "dac4a464f08bfc86d9945d7f2cf303f6"
server_key_iv: "7daf72efde6fc8f3844e5a7d7f774f68"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:31.238102 29435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "24f3f4d0572e4169855cf98f9c4fc4ce"
format_stamp: "Formatted at 2026-08-09 15:16:31 on dist-test-slave-8w9v"
server_key: "dac4a464f08bfc86d9945d7f2cf303f6"
server_key_iv: "7daf72efde6fc8f3844e5a7d7f774f68"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:31.243783 29435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 15:16:31.260294 29453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:31.261862 29435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:16:31.262044 29435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "24f3f4d0572e4169855cf98f9c4fc4ce"
format_stamp: "Formatted at 2026-08-09 15:16:31 on dist-test-slave-8w9v"
server_key: "dac4a464f08bfc86d9945d7f2cf303f6"
server_key_iv: "7daf72efde6fc8f3844e5a7d7f774f68"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:31.262279 29435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:31.296972 29435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:31.301219 29435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:31.301534 29435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:31.302479 29435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:31.303907 29435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:31.303980 29435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:31.304167 29435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:31.304224 29435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 15:16:31.336819 29435 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:44829
I20260809 15:16:31.336859 29576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:44829 every 8 connection(s)
I20260809 15:16:31.338107 29435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:16:31.342661 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 29435
I20260809 15:16:31.342841 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:16:31.343298 15406 external_mini_cluster.cc:1428] Setting key f0ee8e4edaa1d6acf3be775506d929dc
Aug 09 15:16:31 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288591, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:31.357039 29580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:31.340655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:59337 (local address 127.15.11.190:42109)
0809 15:16:31.341123 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:31.341127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:31.341989 (+ 862us) server_negotiation.cc:407] Connection header received
0809 15:16:31.343923 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:31.343929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:31.344028 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:31.344165 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:31.345512 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.346253 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.347783 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.348300 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.348446 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:31.351503 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:31.351507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:31.351510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:31.351551 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:31.354003 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:31.354672 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:31.354678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:31.354682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:31.354809 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:31.355248 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:31.355253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:31.355256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:31.355534 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:31.355684 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:31.356442 (+ 758us) server_negotiation.cc:299] Negotiation successful
0809 15:16:31.356661 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":244,"threads_started":1}
I20260809 15:16:31.358076 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:31.341122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42109 (local address 127.15.11.131:59337)
0809 15:16:31.341813 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:31.341855 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:31.343662 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:31.344405 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:31.344413 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:31.344807 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:31.345351 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:31.345367 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.346414 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.346426 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:31.347638 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:31.347649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.348516 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.348523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:31.348752 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:31.349350 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:31.349371 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:31.351293 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:31.354137 (+ 2844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:31.354144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:31.354157 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:31.354553 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:31.354926 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:31.354931 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:31.354935 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:31.355136 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:31.355658 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:31.355664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:31.355998 (+ 334us) client_negotiation.cc:769] Sending connection context
0809 15:16:31.356330 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 15:16:31.356812 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":271,"threads_started":1}
I20260809 15:16:31.359385 29578 heartbeater.cc:347] Connected to a master server at 127.15.11.190:42109
I20260809 15:16:31.359689 29578 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:31.360453 29578 heartbeater.cc:511] Master 127.15.11.190:42109 requested a full tablet report, sending...
I20260809 15:16:31.361886 29034 ts_manager.cc:194] Registered new tserver with Master: 24f3f4d0572e4169855cf98f9c4fc4ce (127.15.11.131:44829)
I20260809 15:16:31.362727 29034 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:59337
I20260809 15:16:31.370538 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:16:31.372270 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 29123
Aug 09 15:16:31 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:16:31.388891 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:31.373258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.15.11.129:36569)
0809 15:16:31.373848 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:31.373854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:31.373880 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:16:31.373935 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:31.373941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:31.374114 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:31.374231 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:31.375678 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.376824 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.378342 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.378933 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.379077 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:31.383526 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:31.383535 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:31.383548 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:31.383581 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:31.386439 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:31.387072 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:31.387078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:31.387080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:31.387223 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:31.387545 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:31.387548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:31.387550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:31.387850 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:31.388093 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:31.388399 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 15:16:31.388598 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":307,"threads_started":1}
I20260809 15:16:31.390108 29265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:31.472254 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 29123
I20260809 15:16:31.493203 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 29277
Aug 09 15:16:31 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:16:31.509025 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:31.494215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.15.11.130:43153)
0809 15:16:31.494738 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:31.494744 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:31.494769 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:16:31.494838 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:31.494844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:31.495042 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:31.495171 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:31.496425 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.497425 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.498641 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.499171 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.499264 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:31.503302 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:31.503313 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:31.503327 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:31.503357 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:31.506183 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:31.506907 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:31.506921 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:31.506923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:31.507062 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:31.507452 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:31.507457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:31.507460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:31.507817 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:31.508107 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:31.508450 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 15:16:31.508702 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":284,"threads_started":1}
I20260809 15:16:31.510214 29424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:31.590662 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 29277
I20260809 15:16:31.615837 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 29435
Aug 09 15:16:31 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:16:31.632591 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:31.617039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.15.11.131:44829)
0809 15:16:31.617713 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:31.617721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:31.617749 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:16:31.617810 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:31.617816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:31.618039 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:31.618174 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:31.619528 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.620653 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.621925 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:31.622466 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:31.622566 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:31.627057 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:31.627068 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:31.627083 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:31.627136 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:31.630090 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:31.630753 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:31.630760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:31.630762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:31.630885 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:31.631211 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:31.631214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:31.631216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:31.631546 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:31.631804 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:31.632036 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 15:16:31.632303 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":351,"threads_started":1}
I20260809 15:16:31.633950 29572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:31.715198 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 29435
I20260809 15:16:31.735730 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 28993
I20260809 15:16:31.737077 29068 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:31.844959 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 28993
I20260809 15:16:31.875959 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:42109
--webserver_interface=127.15.11.190
--webserver_port=42233
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:42109
--ranger_config_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:44861/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.11.148:44861/kms with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:32.221545 29591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:32.221899 29591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:32.222070 29591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:32.229398 29591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:16:32.229552 29591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:32.229664 29591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:32.229799 29591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:16:32.229892 29591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:16:32.240383 29591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.11.148:44861/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:42109
--ranger_config_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:42109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=42233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:32.241794 29591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:32.243204 29591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:32.256055 29596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:32.256138 29597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:32.256098 29599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:32.257126 29591 server_base.cc:1034] running on GCE node
I20260809 15:16:32.257933 29591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:32.259305 29591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:32.260573 29591 hybrid_clock.cc:648] HybridClock initialized: now 1786288592260505 us; error 63 us; skew 500 ppm
I20260809 15:16:32.260944 29591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:32 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:32.264170 29591 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:16:32.265774 29591 webserver.cc:460] Webserver started at http://127.15.11.190:42233/ using document root <none> and password file <none>
I20260809 15:16:32.266479 29591 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:32.266626 29591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:16:32 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:32.357800 29591 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 15:16:32.361352 29609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:32.362776 29591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:16:32.362977 29591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "f0eec95e269d4fbe8e8365731a5dff6b"
format_stamp: "Formatted at 2026-08-09 15:16:28 on dist-test-slave-8w9v"
server_key: "659c82b913b6e401383ad4bc1163f397"
server_key_iv: "df50a1ef58d3e05401f4ed8a9effe915"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:32.363927 29591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:32.385401 29591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:32.389118 29591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:32.389492 29591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:32.408285 29591 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:42109
I20260809 15:16:32.408304 29671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:42109 every 8 connection(s)
I20260809 15:16:32.409854 29591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:16:32.412338 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 29591
I20260809 15:16:32.413436 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:44861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:36569
--local_ip_for_outbound_sockets=127.15.11.129
--tserver_master_addrs=127.15.11.190:42109
--webserver_port=33871
--webserver_interface=127.15.11.129
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.11.148:44861/kms with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:16:32.419193 29672 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:16:32.424460 29672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Bootstrap starting.
I20260809 15:16:32.444278 29672 log.cc:824] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:32.453550 29672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:16:32.454087 29672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Bootstrap complete.
I20260809 15:16:32.463788 29672 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } }
I20260809 15:16:32.464685 29672 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0eec95e269d4fbe8e8365731a5dff6b, State: Initialized, Role: FOLLOWER
I20260809 15:16:32.465389 29672 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } }
I20260809 15:16:32.465639 29672 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:16:32.465770 29672 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:16:32.466002 29672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:16:32.470140 29672 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } }
I20260809 15:16:32.470607 29672 leader_election.cc:302] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0eec95e269d4fbe8e8365731a5dff6b; no voters:
I20260809 15:16:32.471226 29672 leader_election.cc:288] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:16:32.471427 29677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:16:32.473053 29672 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:16:32.478346 29677 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [term 2 LEADER]: Becoming Leader. State: Replica: f0eec95e269d4fbe8e8365731a5dff6b, State: Running, Role: LEADER
I20260809 15:16:32.478821 29677 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } }
I20260809 15:16:32.481755 29678 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } } }
I20260809 15:16:32.482085 29678 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:32.482551 29679 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0eec95e269d4fbe8e8365731a5dff6b. Latest consensus state: current_term: 2 leader_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0eec95e269d4fbe8e8365731a5dff6b" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42109 } } }
I20260809 15:16:32.482841 29679 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:32.483264 29692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:16:32.484014 29672 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 15:16:32.489692 29692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:16:32.491384 29692 catalog_manager.cc:1259] Loaded cluster ID: 9feb0abcbba84441ac407b2024c97764
I20260809 15:16:32.491497 29692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:16:32.496358 29692 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:16:32.507243 29692 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Loaded TSK: 0
I20260809 15:16:32.509498 29692 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:16:32.895339 29675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:32.895745 29675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:32.895917 29675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:32.907696 29675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:32.907857 29675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:32.908085 29675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:16:32.924221 29675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.11.148:44861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:36569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=33871
--tserver_master_addrs=127.15.11.190:42109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:32.926012 29675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:32.927832 29675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:32.970134 29712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:32.971513 29711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:32.975803 29714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:32.976079 29675 server_base.cc:1034] running on GCE node
I20260809 15:16:32.977082 29675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:32.978971 29675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:32.982481 29675 hybrid_clock.cc:648] HybridClock initialized: now 1786288592982013 us; error 454 us; skew 500 ppm
I20260809 15:16:32.982949 29675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:32 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:32.990038 29675 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:16:33.008603 29675 webserver.cc:460] Webserver started at http://127.15.11.129:33871/ using document root <none> and password file <none>
I20260809 15:16:33.009735 29675 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:33.009944 29675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:16:33 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:33.079910 29675 fs_manager.cc:709] Time spent opening directory manager: real 0.021s user 0.004s sys 0.011s
I20260809 15:16:33.096813 29724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:33.099979 29675 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.005s sys 0.002s
I20260809 15:16:33.100236 29675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "0d6f183adff6439aa158fc2b962c2c48"
format_stamp: "Formatted at 2026-08-09 15:16:29 on dist-test-slave-8w9v"
server_key: "c2151887643ef5c7ded94d55d1e13702"
server_key_iv: "5574cfabdbd25d5850fb4dc82199d3fd"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:33.101651 29675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:33.143015 29675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:33.148526 29675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:33.148818 29675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:33.150094 29675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:33.151563 29675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:33.151648 29675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:33.151785 29675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:33.151865 29675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:33.237100 29675 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:36569
I20260809 15:16:33.237174 29849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:36569 every 8 connection(s)
I20260809 15:16:33.238785 29675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:16:33.245990 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 29675
I20260809 15:16:33.246889 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:44861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:43153
--local_ip_for_outbound_sockets=127.15.11.130
--tserver_master_addrs=127.15.11.190:42109
--webserver_port=37949
--webserver_interface=127.15.11.130
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.11.148:44861/kms with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:16:33 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288592, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:33.270100 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:33.242522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42109 (local address 127.15.11.129:40721)
0809 15:16:33.243418 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:33.243456 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:33.245254 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:33.247408 (+ 2154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:33.247420 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:33.248179 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:33.249005 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:33.249033 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:33.251838 (+ 2805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:33.251844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:33.253926 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:33.253955 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:33.255762 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:33.255771 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:33.256102 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:33.257221 (+ 1119us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:33.257264 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:33.260652 (+ 3388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:33.264625 (+ 3973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:33.264634 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:33.264649 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:33.265235 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:33.265792 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:33.265796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:33.265799 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:33.266042 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:33.267213 (+ 1171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:33.267241 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:33.267710 (+ 469us) client_negotiation.cc:769] Sending connection context
0809 15:16:33.268026 (+ 316us) client_negotiation.cc:240] Negotiation successful
0809 15:16:33.268734 (+ 708us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":349,"threads_started":1}
I20260809 15:16:33.270274 29852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:33.242480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:40721 (local address 127.15.11.190:42109)
0809 15:16:33.243331 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:33.243350 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:33.243579 (+ 229us) server_negotiation.cc:407] Connection header received
0809 15:16:33.246073 (+ 2494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:33.246092 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:33.246567 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:33.246954 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:33.249261 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:33.251605 (+ 2344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:33.254122 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:33.255170 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:33.255356 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:33.260947 (+ 5591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:33.260958 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:33.260981 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:33.261033 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:33.264454 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:33.265421 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:33.265430 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:33.265439 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:33.265649 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:33.266212 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:33.266219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:33.266225 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:33.266979 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:33.267179 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:33.268434 (+ 1255us) server_negotiation.cc:299] Negotiation successful
0809 15:16:33.268858 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":344,"threads_started":1}
I20260809 15:16:33.274425 29850 heartbeater.cc:347] Connected to a master server at 127.15.11.190:42109
I20260809 15:16:33.274797 29850 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:33.275815 29850 heartbeater.cc:511] Master 127.15.11.190:42109 requested a full tablet report, sending...
W20260809 15:16:33.279582 29850 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:42109 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 15:16:33.279834 29850 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:33.280112 29850 heartbeater.cc:511] Master 127.15.11.190:42109 requested a full tablet report, sending...
I20260809 15:16:33.281807 29850 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:33.282167 29850 heartbeater.cc:511] Master 127.15.11.190:42109 requested a full tablet report, sending...
W20260809 15:16:33.283286 29850 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:16:33.666637 29854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:33.666961 29854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:33.667127 29854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:33.674988 29854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:33.675124 29854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:33.675310 29854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:16:33.685576 29854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.11.148:44861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:43153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=37949
--tserver_master_addrs=127.15.11.190:42109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:33.687023 29854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:33.688433 29854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:33.707170 29870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:33.708384 29872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:33.713351 29875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:33.713436 29854 server_base.cc:1034] running on GCE node
I20260809 15:16:33.714174 29854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:33.715440 29854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:33.716684 29854 hybrid_clock.cc:648] HybridClock initialized: now 1786288593716622 us; error 51 us; skew 500 ppm
I20260809 15:16:33.717104 29854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:33 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786288593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 15:16:33 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:33.721367 29854 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:16:33.723364 29854 webserver.cc:460] Webserver started at http://127.15.11.130:37949/ using document root <none> and password file <none>
I20260809 15:16:33.724318 29854 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:33.724491 29854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:16:33 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:33.757247 29854 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:16:33.761577 29883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:33.764340 29854 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:16:33.764529 29854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "d5cb73066ba248ac9d057f740b862b9f"
format_stamp: "Formatted at 2026-08-09 15:16:30 on dist-test-slave-8w9v"
server_key: "e6465b970a28c3154d58d434470fc00e"
server_key_iv: "01aa2e626358cd602dcfba6ed6db104b"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:33.765906 29854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:33.789106 29854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:33.794217 29854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:33.794560 29854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:33.795814 29854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:33.797719 29854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:33.797860 29854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:33.797986 29854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:33.798171 29854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:33.854700 29854 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:43153
I20260809 15:16:33.854799 30005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:43153 every 8 connection(s)
I20260809 15:16:33.856674 29854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:16:33.866251 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 29854
I20260809 15:16:33.867347 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.11.148:44861/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:44829
--local_ip_for_outbound_sockets=127.15.11.131
--tserver_master_addrs=127.15.11.190:42109
--webserver_port=36119
--webserver_interface=127.15.11.131
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:41929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.11.148:44861/kms with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:16:33 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:33.894480 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:33.859845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:48391 (local address 127.15.11.190:42109)
0809 15:16:33.873404 (+ 13559us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:33.873411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:33.873436 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:16:33.875219 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:33.875225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:33.875328 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:33.875441 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:33.877812 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:33.878979 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:33.881510 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:33.882134 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:33.882243 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:33.886885 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:33.886892 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:33.886897 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:33.886939 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:33.890661 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:33.891588 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:33.891593 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:33.891596 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:33.891745 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:33.892351 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:33.892356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:33.892359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:33.892723 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:33.892857 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:33.893855 (+ 998us) server_negotiation.cc:299] Negotiation successful
0809 15:16:33.894077 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13385,"thread_start_us":13223,"threads_started":1}
I20260809 15:16:33.897678 30009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:33.860364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42109 (local address 127.15.11.130:48391)
0809 15:16:33.871225 (+ 10861us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:33.871289 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:33.874695 (+ 3406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:33.875889 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:33.875902 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:33.876642 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:33.877580 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:33.877608 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:33.879217 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:33.879223 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:33.881336 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:33.881351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:33.882622 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:33.882630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:33.882933 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:33.883744 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:33.883775 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:33.886623 (+ 2848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:33.890827 (+ 4204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:33.890836 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:33.890852 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:33.891445 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:33.891908 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:33.891913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:33.891917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:33.892196 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:33.892955 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:33.892962 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:33.893394 (+ 432us) client_negotiation.cc:769] Sending connection context
0809 15:16:33.894969 (+ 1575us) client_negotiation.cc:240] Negotiation successful
0809 15:16:33.896340 (+ 1371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9580,"thread_start_us":9828,"threads_started":1}
I20260809 15:16:33.899406 30006 heartbeater.cc:347] Connected to a master server at 127.15.11.190:42109
I20260809 15:16:33.899732 30006 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:33.900710 30006 heartbeater.cc:511] Master 127.15.11.190:42109 requested a full tablet report, sending...
I20260809 15:16:33.902551 29631 ts_manager.cc:194] Registered new tserver with Master: d5cb73066ba248ac9d057f740b862b9f (127.15.11.130:43153)
I20260809 15:16:33.904426 29631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:48391
Aug 09 15:16:34 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:34.283492 29850 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:34.284015 29850 heartbeater.cc:511] Master 127.15.11.190:42109 requested a full tablet report, sending...
I20260809 15:16:34.285467 29631 ts_manager.cc:194] Registered new tserver with Master: 0d6f183adff6439aa158fc2b962c2c48 (127.15.11.129:36569)
I20260809 15:16:34.286672 29631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:40721
W20260809 15:16:34.377521 30010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:34.378019 30010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:34.378224 30010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:34.389945 30010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:34.390159 30010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:34.390381 30010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:16:34.402691 30010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:41929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.11.148:44861/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:44829
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=36119
--tserver_master_addrs=127.15.11.190:42109
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:34.404335 30010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:34.406199 30010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:34.469025 30016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:34.468703 30018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:34.471100 30010 server_base.cc:1034] running on GCE node
W20260809 15:16:34.471304 30021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:34.472328 30010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:34.489868 30010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:34.491255 30010 hybrid_clock.cc:648] HybridClock initialized: now 1786288594491163 us; error 86 us; skew 500 ppm
I20260809 15:16:34.491786 30010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:34 dist-test-slave-8w9v krb5kdc[28492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288594, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:34.496308 30010 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:16:34.527220 30010 webserver.cc:460] Webserver started at http://127.15.11.131:36119/ using document root <none> and password file <none>
I20260809 15:16:34.528556 30010 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:34.528818 30010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:16:34 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288594, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:34.572305 30010 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.006s sys 0.005s
I20260809 15:16:34.582937 30028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:34.585486 30010 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.006s sys 0.004s
I20260809 15:16:34.585757 30010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "24f3f4d0572e4169855cf98f9c4fc4ce"
format_stamp: "Formatted at 2026-08-09 15:16:31 on dist-test-slave-8w9v"
server_key: "dac4a464f08bfc86d9945d7f2cf303f6"
server_key_iv: "7daf72efde6fc8f3844e5a7d7f774f68"
server_key_version: "kuduclusterkey@0"
I20260809 15:16:34.587356 30010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 15:16:34 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288593, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for HTTP/127.15.11.148@KRBTEST.COM
I20260809 15:16:34.635039 30010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:34.647104 30010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:34.647524 30010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:34.650116 30010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:34.657137 30010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:34.657317 30010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 15:16:34.657521 30010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:34.657658 30010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:34.873963 30010 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:44829
I20260809 15:16:34.874032 30157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:44829 every 8 connection(s)
I20260809 15:16:34.876034 30010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:16:34.880720 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 30010
Aug 09 15:16:34 dist-test-slave-8w9v krb5kdc[28492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288594, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:34.903008 30166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.882194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.15.11.190:42109)
0809 15:16:34.885839 (+ 3645us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:34.885843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:34.885858 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:16:34.885900 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:34.885903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:34.885970 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:34.886090 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:34.888188 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.889458 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.891182 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.891861 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.891993 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.895338 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:34.895345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:34.895349 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:34.895384 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:34.900279 (+ 4895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:34.900896 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:34.900903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:34.900909 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:34.901080 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:34.901488 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:34.901494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:34.901497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:34.901797 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:34.902093 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:34.902371 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 15:16:34.902597 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":117,"server-negotiator.queue_time_us":3549,"thread_start_us":2164,"threads_started":1}
I20260809 15:16:34.903379 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.880390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:43543 (local address 127.15.11.190:42109)
0809 15:16:34.884679 (+ 4289us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:34.884689 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:34.884719 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:16:34.884801 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:34.884810 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:34.884933 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:34.885082 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:34.886972 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.888163 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.890238 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.890900 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.891044 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.895310 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:34.895321 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:34.895327 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:34.895370 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:34.900151 (+ 4781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:34.900882 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:34.900889 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:34.900893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:34.901024 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:34.901524 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:34.901530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:34.901533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:34.901872 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:34.902032 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:34.902681 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 15:16:34.902903 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":4120,"spinlock_wait_cycles":2816,"thread_start_us":5215,"threads_started":1}
I20260809 15:16:34.904973 30165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.879757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42109 (local address 127.15.11.131:43543)
0809 15:16:34.880605 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:34.880658 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:34.884310 (+ 3652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:34.885643 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:34.885655 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:34.886185 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:34.886775 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:34.886802 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.888353 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.888361 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:34.890072 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:34.890085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.891161 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.891169 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:34.891457 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.892266 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:34.892288 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:34.895065 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:34.900329 (+ 5264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:34.900336 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:34.900347 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:34.900762 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:34.901184 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:34.901190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:34.901194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:34.901408 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:34.902006 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:34.902014 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:34.902343 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 15:16:34.902606 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 15:16:34.903139 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":1587,"threads_started":1}
I20260809 15:16:34.906450 30159 heartbeater.cc:347] Connected to a master server at 127.15.11.190:42109
I20260809 15:16:34.906744 30159 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:34.907472 30159 heartbeater.cc:511] Master 127.15.11.190:42109 requested a full tablet report, sending...
I20260809 15:16:34.908649 30006 heartbeater.cc:503] Master 127.15.11.190:42109 was elected leader, sending a full tablet report...
I20260809 15:16:34.908885 29631 ts_manager.cc:194] Registered new tserver with Master: 24f3f4d0572e4169855cf98f9c4fc4ce (127.15.11.131:44829)
I20260809 15:16:34.909906 29631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:43543
I20260809 15:16:34.918170 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:16:34.933676 30168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.921268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37852 (local address 127.15.11.190:42109)
0809 15:16:34.921482 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:34.921491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:34.921880 (+ 389us) server_negotiation.cc:407] Connection header received
0809 15:16:34.922017 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:34.922020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:34.922086 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:34.922160 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:34.923651 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.924421 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.925776 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.926182 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.926276 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.928984 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:34.928988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:34.928991 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:34.929018 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:34.931345 (+ 2327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:34.931939 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:34.931943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:34.931945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:34.932083 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:34.932483 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:34.932486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:34.932488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:34.932728 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:34.932927 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:34.933187 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 15:16:34.933350 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:16:34.938310 29631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:16:34.945654 29631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:16:34.972525 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.959238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:44829 (local address 127.0.0.1:59006)
0809 15:16:34.959996 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:34.960051 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:34.960273 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:34.961764 (+ 1491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:34.961775 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:34.961816 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:34.962332 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:34.962353 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.965273 (+ 2920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.965282 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:34.968509 (+ 3227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:34.968532 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.971214 (+ 2682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.971223 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:34.971498 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.971577 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 15:16:34.971733 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 15:16:34.971991 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":351,"threads_started":1}
I20260809 15:16:34.974812 30182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.959627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59006 (local address 127.15.11.131:44829)
0809 15:16:34.961011 (+ 1384us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:34.961023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:34.961064 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:16:34.961184 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:34.961195 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:34.961477 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:34.962588 (+ 1111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:34.962959 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.965076 (+ 2117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.968789 (+ 3713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.970824 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.973608 (+ 2784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.973981 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:34.974072 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 15:16:34.974293 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1137,"thread_start_us":947,"threads_started":1}
I20260809 15:16:34.979156 30080 tablet_service.cc:1467] Processing CreateTablet for tablet fe067abc08094fffb70feda103b510a7 (DEFAULT_TABLE table=test-table [id=c8b0f04651f342adbfec9ffd4d3afa29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:16:34.980810 30080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe067abc08094fffb70feda103b510a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:34.982331 30184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.960401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:43153 (local address 127.0.0.1:40774)
0809 15:16:34.971776 (+ 11375us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:34.971800 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:34.971944 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:34.974466 (+ 2522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:34.974474 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:34.974510 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:34.974983 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:34.974993 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.977947 (+ 2954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.977952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:34.979519 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:34.979533 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.981477 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.981483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:34.981649 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.981687 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:16:34.981812 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 15:16:34.981963 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10740,"thread_start_us":15349,"threads_started":1}
I20260809 15:16:34.982321 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.961124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40774 (local address 127.15.11.130:43153)
0809 15:16:34.973211 (+ 12087us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:34.973219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:34.973246 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:16:34.973338 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:34.973348 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:34.973656 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:34.974151 (+ 495us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:34.976244 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.977778 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.979716 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.981265 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.981432 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.981656 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:34.981854 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 15:16:34.982009 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11888,"thread_start_us":12670,"threads_started":1}
I20260809 15:16:34.982643 30181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.960537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:36569 (local address 127.0.0.1:49744)
0809 15:16:34.972834 (+ 12297us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:34.972857 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:34.972990 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:34.976408 (+ 3418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:34.976413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:34.976436 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:34.976632 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:34.976644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.978328 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.978337 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:34.980203 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:34.980219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.981965 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.981973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:34.982158 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.982190 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:16:34.982265 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 15:16:34.982403 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12165,"thread_start_us":10423,"threads_started":1}
I20260809 15:16:34.983536 30186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:34.961167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49744 (local address 127.15.11.129:36569)
0809 15:16:34.975763 (+ 14596us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:34.975773 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:34.975801 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:16:34.975899 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:34.975905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:34.976194 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:34.976803 (+ 609us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:34.977077 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.978178 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.980390 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:34.981763 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:34.982666 (+ 903us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:34.982913 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:34.983005 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 15:16:34.983166 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14372,"thread_start_us":14148,"threads_started":1}
I20260809 15:16:34.986578 29937 tablet_service.cc:1467] Processing CreateTablet for tablet fe067abc08094fffb70feda103b510a7 (DEFAULT_TABLE table=test-table [id=c8b0f04651f342adbfec9ffd4d3afa29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:16:34.986954 29779 tablet_service.cc:1467] Processing CreateTablet for tablet fe067abc08094fffb70feda103b510a7 (DEFAULT_TABLE table=test-table [id=c8b0f04651f342adbfec9ffd4d3afa29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:16:34.988062 29937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe067abc08094fffb70feda103b510a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:34.988243 29779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe067abc08094fffb70feda103b510a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:34.996935 30188 tablet_bootstrap.cc:492] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: Bootstrap starting.
I20260809 15:16:35.000847 30188 tablet_bootstrap.cc:654] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:35.002178 30189 tablet_bootstrap.cc:492] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: Bootstrap starting.
I20260809 15:16:35.003197 30188 log.cc:824] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:35.003762 30190 tablet_bootstrap.cc:492] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: Bootstrap starting.
I20260809 15:16:35.006247 30189 tablet_bootstrap.cc:654] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:35.007475 30189 log.cc:824] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:35.007515 30190 tablet_bootstrap.cc:654] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:35.007710 30188 tablet_bootstrap.cc:492] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: No bootstrap required, opened a new log
I20260809 15:16:35.008078 30188 ts_tablet_manager.cc:1397] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260809 15:16:35.008807 30190 log.cc:824] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:35.010407 30189 tablet_bootstrap.cc:492] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: No bootstrap required, opened a new log
I20260809 15:16:35.010706 30189 ts_tablet_manager.cc:1397] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 15:16:35.012434 30190 tablet_bootstrap.cc:492] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: No bootstrap required, opened a new log
I20260809 15:16:35.012677 30190 ts_tablet_manager.cc:1397] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 15:16:35.017760 30189 raft_consensus.cc:348] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } }
I20260809 15:16:35.018206 30189 raft_consensus.cc:374] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:35.018311 30189 raft_consensus.cc:729] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d6f183adff6439aa158fc2b962c2c48, State: Initialized, Role: FOLLOWER
I20260809 15:16:35.018445 30188 raft_consensus.cc:348] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } }
I20260809 15:16:35.018850 30188 raft_consensus.cc:374] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:35.018970 30188 raft_consensus.cc:729] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24f3f4d0572e4169855cf98f9c4fc4ce, State: Initialized, Role: FOLLOWER
I20260809 15:16:35.019073 30189 consensus_queue.cc:260] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } }
I20260809 15:16:35.019577 30188 consensus_queue.cc:260] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } }
I20260809 15:16:35.020334 29850 heartbeater.cc:503] Master 127.15.11.190:42109 was elected leader, sending a full tablet report...
I20260809 15:16:35.020836 30189 ts_tablet_manager.cc:1428] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260809 15:16:35.022653 30159 heartbeater.cc:503] Master 127.15.11.190:42109 was elected leader, sending a full tablet report...
I20260809 15:16:35.023138 30188 ts_tablet_manager.cc:1428] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260809 15:16:35.023280 30190 raft_consensus.cc:348] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } }
I20260809 15:16:35.023794 30190 raft_consensus.cc:374] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:35.023950 30190 raft_consensus.cc:729] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5cb73066ba248ac9d057f740b862b9f, State: Initialized, Role: FOLLOWER
I20260809 15:16:35.025552 30190 consensus_queue.cc:260] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } }
I20260809 15:16:35.028187 30190 ts_tablet_manager.cc:1428] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
W20260809 15:16:35.110541 30007 tablet.cc:2369] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:16:35.129119 30196 raft_consensus.cc:482] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:16:35.129150 30160 tablet.cc:2369] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:16:35.129472 30196 raft_consensus.cc:504] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } }
I20260809 15:16:35.131143 30196 leader_election.cc:288] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24f3f4d0572e4169855cf98f9c4fc4ce (127.15.11.131:44829), 0d6f183adff6439aa158fc2b962c2c48 (127.15.11.129:36569)
I20260809 15:16:35.137583 30182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:35.131487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:55427 (local address 127.15.11.131:44829)
0809 15:16:35.131657 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:35.131662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:35.132281 (+ 619us) server_negotiation.cc:407] Connection header received
0809 15:16:35.132537 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:35.132545 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:35.132642 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:35.132732 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:35.133380 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.134041 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.135819 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.136678 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.136858 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:35.136936 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:35.137205 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 15:16:35.137326 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:16:35.137660 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:35.131546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:44829 (local address 127.15.11.130:55427)
0809 15:16:35.132148 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:35.132175 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:35.132344 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:35.132815 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:35.132819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:35.132851 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:35.133219 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:35.133228 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.134170 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.134178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:35.135650 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:35.135662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.136890 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.136896 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:35.137047 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:35.137071 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:16:35.137177 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 15:16:35.137301 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":317,"threads_started":1}
I20260809 15:16:35.138139 30186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:35.132517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:34541 (local address 127.15.11.129:36569)
0809 15:16:35.132746 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:35.132751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:35.133008 (+ 257us) server_negotiation.cc:407] Connection header received
0809 15:16:35.133219 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:35.133224 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:35.133297 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:35.133391 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:35.133896 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.134581 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.136418 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.137149 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.137346 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:35.137422 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:35.137745 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 15:16:35.137899 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 15:16:35.138648 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:35.132342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:36569 (local address 127.15.11.130:34541)
0809 15:16:35.132893 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:35.132905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:35.133072 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:35.133407 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:35.133411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:35.133434 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:35.133733 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:35.133741 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.134734 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.134739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:35.136263 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:35.136276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.137389 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.137395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:35.137544 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:35.137566 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:16:35.138304 (+ 738us) client_negotiation.cc:240] Negotiation successful
0809 15:16:35.138432 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":269,"threads_started":1}
I20260809 15:16:35.138612 30105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe067abc08094fffb70feda103b510a7" candidate_uuid: "d5cb73066ba248ac9d057f740b862b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" is_pre_election: true
I20260809 15:16:35.139150 30105 raft_consensus.cc:2454] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5cb73066ba248ac9d057f740b862b9f in term 0.
I20260809 15:16:35.139605 29802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe067abc08094fffb70feda103b510a7" candidate_uuid: "d5cb73066ba248ac9d057f740b862b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6f183adff6439aa158fc2b962c2c48" is_pre_election: true
I20260809 15:16:35.139830 29885 leader_election.cc:302] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f3f4d0572e4169855cf98f9c4fc4ce, d5cb73066ba248ac9d057f740b862b9f; no voters:
I20260809 15:16:35.140296 30196 raft_consensus.cc:2781] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:16:35.140240 29802 raft_consensus.cc:2454] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5cb73066ba248ac9d057f740b862b9f in term 0.
I20260809 15:16:35.140445 30196 raft_consensus.cc:482] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:16:35.140563 30196 raft_consensus.cc:3037] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:35.142724 30196 raft_consensus.cc:504] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } }
I20260809 15:16:35.143249 30196 leader_election.cc:288] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [CANDIDATE]: Term 1 election: Requested vote from peers 24f3f4d0572e4169855cf98f9c4fc4ce (127.15.11.131:44829), 0d6f183adff6439aa158fc2b962c2c48 (127.15.11.129:36569)
I20260809 15:16:35.143754 30105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe067abc08094fffb70feda103b510a7" candidate_uuid: "d5cb73066ba248ac9d057f740b862b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce"
I20260809 15:16:35.143805 29802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe067abc08094fffb70feda103b510a7" candidate_uuid: "d5cb73066ba248ac9d057f740b862b9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d6f183adff6439aa158fc2b962c2c48"
I20260809 15:16:35.143914 30105 raft_consensus.cc:3037] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:35.144016 29802 raft_consensus.cc:3037] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:35.146049 30105 raft_consensus.cc:2454] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5cb73066ba248ac9d057f740b862b9f in term 1.
I20260809 15:16:35.146294 29802 raft_consensus.cc:2454] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5cb73066ba248ac9d057f740b862b9f in term 1.
I20260809 15:16:35.146425 29885 leader_election.cc:302] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24f3f4d0572e4169855cf98f9c4fc4ce, d5cb73066ba248ac9d057f740b862b9f; no voters:
I20260809 15:16:35.146837 30196 raft_consensus.cc:2781] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:16:35.147250 30196 raft_consensus.cc:686] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 1 LEADER]: Becoming Leader. State: Replica: d5cb73066ba248ac9d057f740b862b9f, State: Running, Role: LEADER
I20260809 15:16:35.147732 30196 consensus_queue.cc:237] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } }
I20260809 15:16:35.152184 29630 catalog_manager.cc:5495] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f reported cstate change: term changed from 0 to 1, leader changed from <none> to d5cb73066ba248ac9d057f740b862b9f (127.15.11.130). New cstate: current_term: 1 leader_uuid: "d5cb73066ba248ac9d057f740b862b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5cb73066ba248ac9d057f740b862b9f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43153 } health_report { overall_health: HEALTHY } } }
I20260809 15:16:35.174490 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:35.169122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40786 (local address 127.15.11.130:43153)
0809 15:16:35.169320 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:35.169324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:35.169452 (+ 128us) server_negotiation.cc:407] Connection header received
0809 15:16:35.169611 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:35.169615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:35.169701 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:35.169815 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:16:35.170272 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.171077 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.172496 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.172913 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.173020 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:35.173481 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:16:35.173994 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:16:35.174098 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:35.174228 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 15:16:35.174287 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:16:35.182330 30105 raft_consensus.cc:1261] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [term 1 FOLLOWER]: Refusing update from remote peer d5cb73066ba248ac9d057f740b862b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:16:35.182355 29802 raft_consensus.cc:1261] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [term 1 FOLLOWER]: Refusing update from remote peer d5cb73066ba248ac9d057f740b862b9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:16:35.183287 30199 consensus_queue.cc:1035] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d6f183adff6439aa158fc2b962c2c48" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 36569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:16:35.183662 30196 consensus_queue.cc:1035] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "24f3f4d0572e4169855cf98f9c4fc4ce" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:16:35.198307 30203 mvcc.cc:205] Tried to move back new op lower bound from 7316638085857218560 to 7316638085728870400. Current Snapshot: MvccSnapshot[applied={T|T < 7316638085857218560}]
I20260809 15:16:35.210355 29630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:16:35.210629 29630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:16:35.214186 29630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0eec95e269d4fbe8e8365731a5dff6b: Sending DeleteTablet for 3 replicas of tablet fe067abc08094fffb70feda103b510a7
I20260809 15:16:35.215432 29937 tablet_service.cc:1514] Processing DeleteTablet for tablet fe067abc08094fffb70feda103b510a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:16:35 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:40774
I20260809 15:16:35.215456 30080 tablet_service.cc:1514] Processing DeleteTablet for tablet fe067abc08094fffb70feda103b510a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:16:35 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:59006
I20260809 15:16:35.216495 29779 tablet_service.cc:1514] Processing DeleteTablet for tablet fe067abc08094fffb70feda103b510a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:16:35 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:49744
I20260809 15:16:35.224488 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 29675
I20260809 15:16:35.225137 30213 tablet_replica.cc:330] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: stopping tablet replica
I20260809 15:16:35.227542 30211 tablet_replica.cc:330] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: stopping tablet replica
I20260809 15:16:35.227857 30213 raft_consensus.cc:2229] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:16:35.227880 30211 raft_consensus.cc:2229] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:16:35.228446 30213 raft_consensus.cc:2258] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:16:35.228644 30211 raft_consensus.cc:2258] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:16:35.229386 30212 tablet_replica.cc:330] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: stopping tablet replica
I20260809 15:16:35.229794 30212 raft_consensus.cc:2229] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:16:35.230270 30212 raft_consensus.cc:2258] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:16:35.231455 30211 ts_tablet_manager.cc:1905] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:16:35.232352 30213 ts_tablet_manager.cc:1905] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:16:35.239327 30211 ts_tablet_manager.cc:1918] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:16:35.239514 30211 log.cc:1196] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/fe067abc08094fffb70feda103b510a7
I20260809 15:16:35.239689 30213 ts_tablet_manager.cc:1918] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:16:35.239850 30213 log.cc:1196] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/fe067abc08094fffb70feda103b510a7
I20260809 15:16:35.240108 30211 ts_tablet_manager.cc:1939] T fe067abc08094fffb70feda103b510a7 P d5cb73066ba248ac9d057f740b862b9f: Deleting consensus metadata
I20260809 15:16:35.242169 30212 ts_tablet_manager.cc:1905] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:16:35.242187 29610 catalog_manager.cc:4841] TS d5cb73066ba248ac9d057f740b862b9f (127.15.11.130:43153): tablet fe067abc08094fffb70feda103b510a7 (table test-table [id=c8b0f04651f342adbfec9ffd4d3afa29]) successfully deleted
I20260809 15:16:35.243666 30213 ts_tablet_manager.cc:1939] T fe067abc08094fffb70feda103b510a7 P 0d6f183adff6439aa158fc2b962c2c48: Deleting consensus metadata
I20260809 15:16:35.245621 29611 catalog_manager.cc:4841] TS 0d6f183adff6439aa158fc2b962c2c48 (127.15.11.129:36569): tablet fe067abc08094fffb70feda103b510a7 (table test-table [id=c8b0f04651f342adbfec9ffd4d3afa29]) successfully deleted
I20260809 15:16:35.249217 30212 ts_tablet_manager.cc:1918] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:16:35.249428 30212 log.cc:1196] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/fe067abc08094fffb70feda103b510a7
I20260809 15:16:35.250037 30212 ts_tablet_manager.cc:1939] T fe067abc08094fffb70feda103b510a7 P 24f3f4d0572e4169855cf98f9c4fc4ce: Deleting consensus metadata
I20260809 15:16:35.251861 29611 catalog_manager.cc:4841] TS 24f3f4d0572e4169855cf98f9c4fc4ce (127.15.11.131:44829): tablet fe067abc08094fffb70feda103b510a7 (table test-table [id=c8b0f04651f342adbfec9ffd4d3afa29]) successfully deleted
I20260809 15:16:35.252419 30186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:35.225498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.15.11.129:36569)
0809 15:16:35.226314 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:35.226323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:35.226346 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:16:35.226434 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:35.226442 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:35.226554 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:35.226671 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:35.229257 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.230333 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.232314 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.232966 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.233084 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:35.243439 (+ 10355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:35.243455 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:35.243474 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:35.243524 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:35.247993 (+ 4469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:35.248821 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:35.248831 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:35.248835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:35.249003 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:35.249533 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:35.249539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:35.249542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:35.249998 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:35.251441 (+ 1443us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:35.251588 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 15:16:35.251987 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632}
I20260809 15:16:35.253168 29845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:35.344564 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 29675
I20260809 15:16:35.375240 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 29854
I20260809 15:16:35.389846 30185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:35.376159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40798 (local address 127.15.11.130:43153)
0809 15:16:35.376352 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:35.376357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:35.376448 (+ 91us) server_negotiation.cc:407] Connection header received
0809 15:16:35.376654 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:35.376658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:35.376733 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:35.376862 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:35.378193 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.379112 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.380463 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.380913 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.381048 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:35.383625 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:35.383642 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:35.383659 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:35.383697 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:35.386940 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:35.387602 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:35.387618 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:35.387620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:35.387761 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:35.388162 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:35.388168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:35.388191 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:35.388554 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:35.388873 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:35.389177 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 15:16:35.389444 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:16:35.390810 30000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:35.517447 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 29854
I20260809 15:16:35.541262 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 30010
I20260809 15:16:35.554180 30182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:35.542068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59018 (local address 127.15.11.131:44829)
0809 15:16:35.542302 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:35.542307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:35.542341 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:16:35.542540 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:35.542544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:35.542606 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:35.542688 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:35.543947 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.544775 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.545996 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:35.546398 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:35.546491 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:35.549191 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:35.549203 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:35.549219 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:35.549265 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:35.551829 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:35.552492 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:35.552499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:35.552501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:35.552621 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:35.552914 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:35.552917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:35.552919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:35.553241 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:35.553476 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:35.553671 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 15:16:35.553819 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:16:35.554827 30151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:35.642047 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 30010
I20260809 15:16:35.664402 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 29591
I20260809 15:16:35.665637 29667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:35.782545 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 29591
I20260809 15:16:35.811954 15406 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 15:16:36.156569 15406 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 15:16:36.156774 15406 mini_ranger.cc:66] Stopping Ranger...
I20260809 15:16:36.501417 15406 mini_ranger.cc:68] Stopped Ranger
2026-08-09 15:16:36.501 UTC [28528] LOG: received smart shutdown request
2026-08-09 15:16:36.504 UTC [28528] LOG: background worker "logical replication launcher" (PID 28538) exited with exit code 1
2026-08-09 15:16:36.505 UTC [28533] LOG: shutting down
2026-08-09 15:16:36.658 UTC [28528] LOG: database system is shut down
2026-08-09T15:16:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52450 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:16:36 dist-test-slave-8w9v krb5kdc[30229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:16:36 dist-test-slave-8w9v krb5kdc[30229](info): set up 2 sockets
Aug 09 15:16:36 dist-test-slave-8w9v krb5kdc[30229](info): commencing operation
krb5kdc: starting...
W20260809 15:16:37.930967 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:16:38 dist-test-slave-8w9v krb5kdc[30229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:16:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:16:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:16:38.184291 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37143
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:32903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:37143,127.15.11.189:37767,127.15.11.188:33577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:38.524616 30242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:38.524941 30242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:38.525128 30242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:38.532327 30242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:16:38.532444 30242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:38.532560 30242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:38.532668 30242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:16:38.532763 30242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:16:38.542359 30242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:32903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:37143,127.15.11.189:37767,127.15.11.188:33577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:38.543706 30242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:38.545141 30242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:38.557509 30250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:38.557780 30249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:38.558239 30242 server_base.cc:1034] running on GCE node
W20260809 15:16:38.558346 30252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:38.559177 30242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:38.560513 30242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:38.561710 30242 hybrid_clock.cc:648] HybridClock initialized: now 1786288598561669 us; error 30 us; skew 500 ppm
I20260809 15:16:38.562049 30242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:38 dist-test-slave-8w9v krb5kdc[30229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288598, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:38.569667 30242 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:16:38.571058 30242 webserver.cc:460] Webserver started at http://127.15.11.190:39957/ using document root <none> and password file <none>
I20260809 15:16:38.571702 30242 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:38.571830 30242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:38.572196 30242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:16:38.575089 30242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "0f140d1f3ede4380ac9be80ba60f074c"
format_stamp: "Formatted at 2026-08-09 15:16:38 on dist-test-slave-8w9v"
server_key: "d6dcd4b2a6c549efc9f44423502e18eb"
server_key_iv: "55f497e5f1ada04ce760a981d3ff8f33"
server_key_version: "encryptionkey@0"
I20260809 15:16:38.575868 30242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "0f140d1f3ede4380ac9be80ba60f074c"
format_stamp: "Formatted at 2026-08-09 15:16:38 on dist-test-slave-8w9v"
server_key: "d6dcd4b2a6c549efc9f44423502e18eb"
server_key_iv: "55f497e5f1ada04ce760a981d3ff8f33"
server_key_version: "encryptionkey@0"
I20260809 15:16:38.581032 30242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:16:38.584486 30261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:38.585749 30242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:16:38.585930 30242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "0f140d1f3ede4380ac9be80ba60f074c"
format_stamp: "Formatted at 2026-08-09 15:16:38 on dist-test-slave-8w9v"
server_key: "d6dcd4b2a6c549efc9f44423502e18eb"
server_key_iv: "55f497e5f1ada04ce760a981d3ff8f33"
server_key_version: "encryptionkey@0"
I20260809 15:16:38.586120 30242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:38.606001 30242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:38.609818 30242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:38.610196 30242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:38.628989 30242 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:37143
I20260809 15:16:38.629009 30320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:37143 every 8 connection(s)
I20260809 15:16:38.630421 30242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:16:38.634106 30321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:38.638880 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 30242
I20260809 15:16:38.639123 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:16:38.639976 15406 external_mini_cluster.cc:1428] Setting key fcf6fe988cef63c5e3de6e097a0432c1
I20260809 15:16:38.642081 30321 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:16:38 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288598, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:38.665347 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.645750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37143 (local address 127.0.0.1:35352)
0809 15:16:38.646528 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:38.646581 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:38.648555 (+ 1974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:38.649442 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:38.649447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:38.649992 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:38.650334 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:38.650350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:38.651897 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:38.651904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:38.653825 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:38.653842 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:38.654674 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:38.654682 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:38.654932 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:38.655611 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:38.655646 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:38.658239 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:38.661430 (+ 3191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:38.661442 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:38.661453 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:38.661835 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:38.662235 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:38.662241 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:38.662244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:38.662393 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:38.663440 (+ 1047us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:38.663446 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:38.663646 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 15:16:38.663834 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 15:16:38.664271 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":334,"threads_started":1}
I20260809 15:16:38.665444 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.646967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.15.11.190:37143)
0809 15:16:38.647639 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:38.647649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:38.647706 (+ 57us) server_negotiation.cc:407] Connection header received
0809 15:16:38.648898 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:38.648908 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:38.649177 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:38.649414 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:38.650528 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:38.651738 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:38.653959 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:38.654446 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:38.654567 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:38.658464 (+ 3897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:38.658471 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:38.658481 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:38.658508 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:38.661285 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:38.661981 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:38.661987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:38.661989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:38.662129 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:38.662568 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:38.662575 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:38.662581 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:38.663295 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:38.663535 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:38.664056 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 15:16:38.664499 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":440,"thread_start_us":302,"threads_started":1}
WARNING: no policy specified for kudu/127.15.11.189@KRBTEST.COM; defaulting to no policy
I20260809 15:16:38.668697 30321 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:16:38.669664 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.669287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58774)
0809 15:16:38.669458 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:38.669562 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 15:16:38.669914 30263 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.189@KRBTEST.COM" created.
I20260809 15:16:38.671715 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.671269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58786)
0809 15:16:38.671449 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:38.671540 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 15:16:38.671929 30321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:37767: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab.
Entry for principal kudu/127.15.11.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab.
I20260809 15:16:38.700803 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.189:37767
--webserver_interface=127.15.11.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab
--principal=kudu/127.15.11.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:32903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:37143,127.15.11.189:37767,127.15.11.188:33577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:16:38.721336 30321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } attempt: 1
I20260809 15:16:38.722594 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.722143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58790)
0809 15:16:38.722339 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:38.722451 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 15:16:38.723541 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.723202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58802)
0809 15:16:38.723353 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:38.723422 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 15:16:38.723712 30321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:37767: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
I20260809 15:16:38.791968 30321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } attempt: 2
I20260809 15:16:38.793174 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.792682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58818)
0809 15:16:38.792945 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:38.793036 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 15:16:38.794247 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.793930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58830)
0809 15:16:38.794058 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:38.794133 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 15:16:38.794505 30321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:37767: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
I20260809 15:16:38.885754 30321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } attempt: 3
I20260809 15:16:38.887004 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.886575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58838)
0809 15:16:38.886776 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:38.886872 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 15:16:38.888113 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:38.887787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58850)
0809 15:16:38.887919 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:38.887996 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 15:16:38.888365 30321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:37767: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
W20260809 15:16:39.027223 30328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:39.027573 30328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:39.027734 30328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 15:16:39.031617 30321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } attempt: 4
I20260809 15:16:39.032994 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.032339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58862)
0809 15:16:39.032692 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.032815 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260809 15:16:39.034104 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.033658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58878)
0809 15:16:39.033830 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.033950 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 15:16:39.034298 30321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:37767: Network error: Client connection negotiation failed: client connection to 127.15.11.189:37767: connect: Connection refused (error 111)
W20260809 15:16:39.035516 30328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:16:39.035629 30328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:39.035717 30328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:39.035815 30328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:16:39.035892 30328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:16:39.045426 30328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:32903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:37143,127.15.11.189:37767,127.15.11.188:33577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab
--principal=kudu/127.15.11.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.189:37767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.11.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:39.046727 30328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:39.048094 30328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:39.059818 30333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:39.059890 30337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:39.059921 30334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:39.065625 30328 server_base.cc:1034] running on GCE node
I20260809 15:16:39.066346 30328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:39.067636 30328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:39.068830 30328 hybrid_clock.cc:648] HybridClock initialized: now 1786288599068774 us; error 44 us; skew 500 ppm
I20260809 15:16:39.069136 30328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:39.072384 30328 init.cc:377] Logged in from keytab as kudu/127.15.11.189@KRBTEST.COM (short username kudu)
I20260809 15:16:39.073992 30328 webserver.cc:460] Webserver started at http://127.15.11.189:43167/ using document root <none> and password file <none>
I20260809 15:16:39.074731 30328 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:39.074879 30328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:39.075208 30328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:16:39.078176 30328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/data/instance:
uuid: "9c2f26bb8d9744bc86348e72657b06e8"
format_stamp: "Formatted at 2026-08-09 15:16:39 on dist-test-slave-8w9v"
server_key: "3e66f6ae55530fccea456c194e377afd"
server_key_iv: "b190f0a8c8548b4427454192bdff0e78"
server_key_version: "encryptionkey@0"
I20260809 15:16:39.079043 30328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/wal/instance:
uuid: "9c2f26bb8d9744bc86348e72657b06e8"
format_stamp: "Formatted at 2026-08-09 15:16:39 on dist-test-slave-8w9v"
server_key: "3e66f6ae55530fccea456c194e377afd"
server_key_iv: "b190f0a8c8548b4427454192bdff0e78"
server_key_version: "encryptionkey@0"
I20260809 15:16:39.085269 30328 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 15:16:39.088541 30345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:39.089835 30328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:16:39.090034 30328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/wal
uuid: "9c2f26bb8d9744bc86348e72657b06e8"
format_stamp: "Formatted at 2026-08-09 15:16:39 on dist-test-slave-8w9v"
server_key: "3e66f6ae55530fccea456c194e377afd"
server_key_iv: "b190f0a8c8548b4427454192bdff0e78"
server_key_version: "encryptionkey@0"
I20260809 15:16:39.090224 30328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:39.102790 30328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:39.106971 30328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:39.107190 30328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:39.125347 30328 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.189:37767
I20260809 15:16:39.125384 30402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.189:37767 every 8 connection(s)
I20260809 15:16:39.126667 30328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
I20260809 15:16:39.130398 30404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:39.134135 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 30328
I20260809 15:16:39.134325 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-1/wal/instance
I20260809 15:16:39.134922 15406 external_mini_cluster.cc:1428] Setting key 144cdc847f7925e6c06f4633641d50d7
I20260809 15:16:39.138628 30404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:39.160348 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.142023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35364 (local address 127.15.11.190:37143)
0809 15:16:39.142241 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.142248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.143402 (+ 1154us) server_negotiation.cc:407] Connection header received
0809 15:16:39.145394 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.145402 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.145497 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.145615 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.147622 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.148313 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.150670 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.151069 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.151159 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.154695 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.154699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.154702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.154733 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.157371 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.158084 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.158091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.158094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.158214 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.158627 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.158632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.158635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.158928 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.159086 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.159763 (+ 677us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.160011 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:16:39.161265 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.142418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37143 (local address 127.0.0.1:35364)
0809 15:16:39.143212 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.143264 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:39.145127 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:39.145899 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:39.145912 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:39.146638 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:39.147432 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.147460 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.148462 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.148471 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.150514 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.150531 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.151290 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.151300 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.151605 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.152418 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:39.152439 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:39.154465 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:39.157531 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.157539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.157552 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.157963 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.158327 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.158332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.158335 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.158514 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.159063 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:39.159071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:39.159393 (+ 322us) client_negotiation.cc:769] Sending connection context
0809 15:16:39.159667 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 15:16:39.160119 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":355,"threads_started":1}
WARNING: no policy specified for kudu/127.15.11.188@KRBTEST.COM; defaulting to no policy
I20260809 15:16:39.162925 30404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.188@KRBTEST.COM" created.
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
I20260809 15:16:39.178022 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.163855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.15.11.189:37767)
0809 15:16:39.164465 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.164473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.164503 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:16:39.164555 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.164562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.164785 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.164929 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.165552 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.167165 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.168219 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.169124 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.169251 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.172145 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.172160 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.172177 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.172221 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.175019 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.175838 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.175843 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.175845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.175953 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.176465 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.176468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.176470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.176775 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.176923 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.177394 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.177727 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":134,"server-negotiator.queue_time_us":427,"thread_start_us":241,"threads_started":1}
I20260809 15:16:39.178103 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.163608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58880)
0809 15:16:39.163809 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.163824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:39.163904 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:39.164934 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:39.164937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:39.165266 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:39.165403 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.165411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.167288 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.167291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.168048 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.168057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.169281 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.169286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.169457 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.169984 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:39.169999 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:39.171923 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:39.175166 (+ 3243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.175173 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.175179 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.175714 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.176096 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.176103 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.176110 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.176333 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.176921 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:39.176928 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:39.177114 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 15:16:39.177274 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 15:16:39.177594 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":32}
I20260809 15:16:39.180316 30404 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:16:39.181437 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.180909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43538)
0809 15:16:39.181109 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.181224 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 15:16:39.181696 30347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
I20260809 15:16:39.183284 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.182848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43542)
0809 15:16:39.183023 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.183141 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 15:16:39.183620 30404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:33577: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab.
Entry for principal kudu/127.15.11.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab.
I20260809 15:16:39.195760 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.188:33577
--webserver_interface=127.15.11.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab
--principal=kudu/127.15.11.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:32903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:37143,127.15.11.189:37767,127.15.11.188:33577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:16:39.232983 30404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } attempt: 1
I20260809 15:16:39.234211 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.233653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43556)
0809 15:16:39.233888 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.234027 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 15:16:39.235427 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.234994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43572)
0809 15:16:39.235176 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.235268 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 15:16:39.235666 30404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:33577: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
I20260809 15:16:39.303912 30404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } attempt: 2
I20260809 15:16:39.305230 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.304678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43588)
0809 15:16:39.304918 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.305035 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 15:16:39.306272 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.305882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43598)
0809 15:16:39.306037 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.306120 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 15:16:39.306504 30404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:33577: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
I20260809 15:16:39.333570 30321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } attempt: 5
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288598, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
I20260809 15:16:39.348074 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.334328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58886)
0809 15:16:39.334577 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.334598 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:39.334767 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:39.335160 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:39.335164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:39.335541 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:39.335723 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.335731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.337215 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.337219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.338052 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.338062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.339169 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.339174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.339313 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.339967 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:39.339984 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:39.342026 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:39.345818 (+ 3792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.345822 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.345824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.346157 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.346591 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.346594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.346597 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.346714 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.347355 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:39.347358 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:39.347501 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 15:16:39.347617 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 15:16:39.347827 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 15:16:39.348308 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.334577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58886 (local address 127.15.11.189:37767)
0809 15:16:39.334805 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.334811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.334834 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:16:39.334920 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.334925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.335006 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.335118 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.335910 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.337080 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.338211 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.339013 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.339168 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.342221 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.342228 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.342233 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.342274 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.345677 (+ 3403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.346283 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.346289 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.346295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.346474 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.346826 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.346834 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.346839 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.347243 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.347418 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.347761 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.347985 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 15:16:39.349383 30321 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:16:39.350346 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.350012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43608)
0809 15:16:39.350153 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.350224 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 15:16:39.351212 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.350924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43624)
0809 15:16:39.351052 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.351129 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 15:16:39.351430 30321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:33577: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
I20260809 15:16:39.397732 30404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } attempt: 3
I20260809 15:16:39.398938 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.398390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43626)
0809 15:16:39.398609 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.398734 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 15:16:39.400260 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.399712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43628)
0809 15:16:39.399952 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.400047 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260809 15:16:39.400482 30404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:33577: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
I20260809 15:16:39.402626 30321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } attempt: 1
I20260809 15:16:39.403568 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.403185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43638)
0809 15:16:39.403346 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.403429 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 15:16:39.404534 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.404207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43644)
0809 15:16:39.404349 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.404442 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 15:16:39.404734 30321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:33577: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
I20260809 15:16:39.472946 30321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } attempt: 2
I20260809 15:16:39.474105 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.473596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43648)
0809 15:16:39.473830 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.473950 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 15:16:39.475055 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.474747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43654)
0809 15:16:39.474883 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.474967 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 15:16:39.475261 30321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:33577: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
W20260809 15:16:39.532612 30414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:39.532964 30414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:39.533123 30414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:39.540163 30414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:16:39.540300 30414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:39.540407 30414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:39.540524 30414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:16:39.540612 30414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:16:39.543716 30404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } attempt: 4
I20260809 15:16:39.544942 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.544392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43664)
0809 15:16:39.544640 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.544757 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 15:16:39.545980 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.545602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43670)
0809 15:16:39.545758 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.545843 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 15:16:39.546192 30404 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:33577: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
I20260809 15:16:39.551986 30414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:32903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:37143,127.15.11.189:37767,127.15.11.188:33577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab
--principal=kudu/127.15.11.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.188:33577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.11.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:39.553349 30414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:39.554669 30414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:39.566526 30422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:39.566565 30424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:39.566628 30421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:39.567414 30414 server_base.cc:1034] running on GCE node
I20260809 15:16:39.568078 30414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:39.569389 30414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:39.570616 30414 hybrid_clock.cc:648] HybridClock initialized: now 1786288599570578 us; error 46 us; skew 500 ppm
I20260809 15:16:39.570976 30414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:39.574041 30414 init.cc:377] Logged in from keytab as kudu/127.15.11.188@KRBTEST.COM (short username kudu)
I20260809 15:16:39.575489 30414 webserver.cc:460] Webserver started at http://127.15.11.188:36231/ using document root <none> and password file <none>
I20260809 15:16:39.576234 30414 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:39.576383 30414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:39.576684 30414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:16:39.579537 30414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/data/instance:
uuid: "07242d321fdf4011b07f82a2c65b5785"
format_stamp: "Formatted at 2026-08-09 15:16:39 on dist-test-slave-8w9v"
server_key: "77c7b40634a817494d935f090e7f3fd3"
server_key_iv: "0f53c337eee4bc9667cd477587d8940e"
server_key_version: "encryptionkey@0"
I20260809 15:16:39.580569 30414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/wal/instance:
uuid: "07242d321fdf4011b07f82a2c65b5785"
format_stamp: "Formatted at 2026-08-09 15:16:39 on dist-test-slave-8w9v"
server_key: "77c7b40634a817494d935f090e7f3fd3"
server_key_iv: "0f53c337eee4bc9667cd477587d8940e"
server_key_version: "encryptionkey@0"
I20260809 15:16:39.581442 30321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } attempt: 3
I20260809 15:16:39.582454 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.581976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43686)
0809 15:16:39.582189 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.582290 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 15:16:39.583357 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.583033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43698)
0809 15:16:39.583208 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.583268 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 15:16:39.583546 30321 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:33577: Network error: Client connection negotiation failed: client connection to 127.15.11.188:33577: connect: Connection refused (error 111)
I20260809 15:16:39.585793 30414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:16:39.589217 30434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:39.590570 30414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:16:39.590763 30414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/wal
uuid: "07242d321fdf4011b07f82a2c65b5785"
format_stamp: "Formatted at 2026-08-09 15:16:39 on dist-test-slave-8w9v"
server_key: "77c7b40634a817494d935f090e7f3fd3"
server_key_iv: "0f53c337eee4bc9667cd477587d8940e"
server_key_version: "encryptionkey@0"
I20260809 15:16:39.590963 30414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:39.606343 30414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:39.610239 30414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:39.610531 30414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:39.628826 30414 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.188:33577
I20260809 15:16:39.628818 30493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.188:33577 every 8 connection(s)
I20260809 15:16:39.630573 30414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
I20260809 15:16:39.634315 30494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:39.640177 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 30414
I20260809 15:16:39.640367 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786288392170773-15406-0/minicluster-data/master-2/wal/instance
I20260809 15:16:39.640818 15406 external_mini_cluster.cc:1428] Setting key 5ded9e2c1e823d6367b97523245515f9
I20260809 15:16:39.640692 30494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
I20260809 15:16:39.661768 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.643547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35374 (local address 127.15.11.190:37143)
0809 15:16:39.643754 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.643760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.644536 (+ 776us) server_negotiation.cc:407] Connection header received
0809 15:16:39.645549 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.645556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.645636 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.645771 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.647498 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.648603 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.650529 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.651074 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.651193 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.655122 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.655127 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.655131 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.655162 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.658226 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.659132 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.659136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.659139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.659271 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.659740 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.659743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.659744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.660008 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.660185 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.661098 (+ 913us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.661368 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:16:39.662847 30499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.643535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37143 (local address 127.0.0.1:35374)
0809 15:16:39.644378 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.644426 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:39.645300 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:39.645846 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:39.645852 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:39.646710 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:39.647309 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.647341 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.648752 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.648755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.650384 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.650399 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.651284 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.651289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.651461 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.652201 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:39.652238 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:39.654911 (+ 2673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:39.658360 (+ 3449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.658372 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.658387 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.659007 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.659387 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.659392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.659396 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.659636 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.660177 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:39.660188 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:39.660678 (+ 490us) client_negotiation.cc:769] Sending connection context
0809 15:16:39.661014 (+ 336us) client_negotiation.cc:240] Negotiation successful
0809 15:16:39.661467 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":440,"threads_started":1}
I20260809 15:16:39.663053 30497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.642612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43706 (local address 127.15.11.188:33577)
0809 15:16:39.643366 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.643376 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.643428 (+ 52us) server_negotiation.cc:407] Connection header received
0809 15:16:39.645290 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.645310 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.645587 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.645807 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.647643 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.649084 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.650792 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.651528 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.651623 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.656241 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.656250 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.656263 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.656291 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.659636 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.660584 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.660591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.660593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.660738 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.661190 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.661195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.661198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.661660 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.661967 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.662331 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.662570 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":335,"threads_started":1}
I20260809 15:16:39.664541 30494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:39.677780 30499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.665195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:37767 (local address 127.0.0.1:58894)
0809 15:16:39.665684 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.665701 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:39.665831 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:39.666244 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:39.666248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:39.666605 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:39.666746 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.666753 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.667708 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.667715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.668683 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.668695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.669391 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.669397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.669622 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.670165 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:39.670181 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:39.672022 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:39.675397 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.675403 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.675407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.675762 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.676185 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.676190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.676193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.676327 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.676826 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:39.676831 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:39.676975 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 15:16:39.677112 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 15:16:39.677406 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377}
I20260809 15:16:39.677824 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.665466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.15.11.189:37767)
0809 15:16:39.665658 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.665663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.665775 (+ 112us) server_negotiation.cc:407] Connection header received
0809 15:16:39.665967 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.665972 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.666054 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.666176 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.666900 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.667577 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.668824 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.669240 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.669376 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.672248 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.672253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.672256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.672287 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.675267 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.675877 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.675883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.675886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.676045 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.676463 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.676469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.676472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.676711 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.676857 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.677210 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.677436 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:16:39.679008 30494 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:16:39.680629 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.666073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.15.11.190:37143)
0809 15:16:39.666291 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.666297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.666312 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:16:39.666358 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.666361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.666412 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.666511 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.667685 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.668470 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.669695 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.670315 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.670453 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.674808 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.674813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.674816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.674845 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.677403 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.678183 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.678188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.678191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.678351 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.678882 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.678886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.678888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.679204 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.679514 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.680008 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.680299 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
I20260809 15:16:39.692871 30499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.679640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43722)
0809 15:16:39.679942 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.679981 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:39.680086 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:39.680451 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:39.680455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:39.680881 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:39.681056 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.681067 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.682037 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.682042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.683205 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.683217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.684234 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.684239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.684491 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.685002 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:39.685023 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:39.687225 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:39.690331 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.690336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.690340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.690836 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.691180 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.691185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.691188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.691351 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.691916 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:39.691920 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:39.692033 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 15:16:39.692236 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 15:16:39.692541 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":5}
I20260809 15:16:39.693156 30497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.679753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.15.11.188:33577)
0809 15:16:39.679925 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.679931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.680034 (+ 103us) server_negotiation.cc:407] Connection header received
0809 15:16:39.680222 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.680228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.680294 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.680390 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.681196 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.681913 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.683355 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.684040 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.684235 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.687444 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.687450 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.687454 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.687489 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.690179 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.690958 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.690962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.690964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.691067 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.691469 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.691472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.691473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.691790 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.691945 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.692335 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.692764 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":162,"server-negotiator.queue_time_us":49}
I20260809 15:16:39.697813 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.682209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.15.11.189:37767)
0809 15:16:39.682360 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.682366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.682430 (+ 64us) server_negotiation.cc:407] Connection header received
0809 15:16:39.682607 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.682612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.682671 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.682764 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.684254 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.685318 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.686803 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.687217 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.687349 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.691765 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.691771 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.691775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.691808 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.695096 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.695795 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.695803 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.695807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.695950 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.696384 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.696397 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.696400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.696667 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.696951 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.697252 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.697508 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:16:39.698356 30494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785: Bootstrap starting.
I20260809 15:16:39.702138 30494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:39.702203 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:16:39.703167 30494 log.cc:824] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:39.705853 30494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785: No bootstrap required, opened a new log
I20260809 15:16:39.714206 30494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } }
I20260809 15:16:39.714733 30494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:39.714919 30494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07242d321fdf4011b07f82a2c65b5785, State: Initialized, Role: FOLLOWER
I20260809 15:16:39.715718 30494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } }
I20260809 15:16:39.717087 30509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:39.717566 30509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:16:39.719347 30494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:16:39.727761 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.707092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.15.11.190:37143)
0809 15:16:39.707287 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.707295 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.707820 (+ 525us) server_negotiation.cc:407] Connection header received
0809 15:16:39.708112 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.708120 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.708245 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.708335 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.711208 (+ 2873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.712323 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.714552 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.715222 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.715331 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.719706 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.719713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.719717 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.719759 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.723379 (+ 3620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.724449 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.724455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.724461 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.724638 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.725163 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.725168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.725172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.725546 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.725807 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.727103 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.727361 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:16:39.732890 30409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.708000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.15.11.189:37767)
0809 15:16:39.708171 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.708177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.712705 (+ 4528us) server_negotiation.cc:407] Connection header received
0809 15:16:39.712968 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.712974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.713048 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.713114 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.716032 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.717100 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.719464 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.720105 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.720235 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.724896 (+ 4661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.724906 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.724914 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.724972 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.729251 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.730175 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.730182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.730188 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.730343 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.730802 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.730806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.730809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.731189 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.731400 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.732032 (+ 632us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.732437 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:16:39.735442 30497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.712729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43736 (local address 127.15.11.188:33577)
0809 15:16:39.712961 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.712966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.716696 (+ 3730us) server_negotiation.cc:407] Connection header received
0809 15:16:39.716970 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.716977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.717055 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.717129 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.719414 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.720583 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.722745 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.723497 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.723580 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.727664 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.727669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.727674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.727710 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.731472 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.732410 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.732415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.732419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.732691 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.733161 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.733165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.733167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.733574 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.734745 (+ 1171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.734875 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.735066 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 15:16:39.740471 30521 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:16:39.740599 30521 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:16:39.737454 30414 master_runner.cc:186] Error getting master registration for 127.15.11.190:37143: OK, instance_id { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" instance_seqno: 1786288598623452 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:16:39.747665 30414 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:16:39.760730 30321 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } attempt: 4
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288598, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
I20260809 15:16:39.776522 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.761314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43752)
0809 15:16:39.761520 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.761537 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:39.761626 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:39.761935 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:39.761939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:39.762382 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:39.762543 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.762552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.764027 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.764034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.765525 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.765541 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.766261 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.766267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.766485 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.767075 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:39.767101 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:39.769605 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:39.773583 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.773590 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.773597 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.774133 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.774656 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.774660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.774664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.774836 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.775457 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:39.775461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:39.775622 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 15:16:39.775765 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 15:16:39.776065 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 15:16:39.776522 30497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.761398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43752 (local address 127.15.11.188:33577)
0809 15:16:39.761579 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.761586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.761607 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:16:39.761721 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.761726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.761799 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.761919 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.762680 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.763851 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.765676 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.766117 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.766205 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.769788 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.769795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.769799 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.769834 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.773441 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.774294 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.774300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.774303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.774514 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.774961 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.774966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.774971 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.775333 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.775494 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.775860 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.776098 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:16:39.782495 30321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c: Bootstrap starting.
I20260809 15:16:39.786674 30321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:39.788093 30321 log.cc:824] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:39.791345 30321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c: No bootstrap required, opened a new log
I20260809 15:16:39.802421 30321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } }
I20260809 15:16:39.802811 30321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:39.802932 30321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f140d1f3ede4380ac9be80ba60f074c, State: Initialized, Role: FOLLOWER
I20260809 15:16:39.803601 30321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } }
I20260809 15:16:39.804657 30524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:39.805105 30524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:16:39.806001 30321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:16:39.819131 30535 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:16:39.819358 30535 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:16:39.815368 30242 master_runner.cc:186] Error getting master registration for 127.15.11.189:37767: OK, instance_id { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" instance_seqno: 1786288599120201 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:16:39.825587 30242 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:16:39.845405 30404 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } attempt: 5
Aug 09 15:16:39 dist-test-slave-8w9v krb5kdc[30229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288599, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
I20260809 15:16:39.859776 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.846022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:33577 (local address 127.0.0.1:43762)
0809 15:16:39.846332 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:39.846354 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:39.846489 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:39.846835 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:39.846839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:39.847284 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:39.847450 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.847459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.848585 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.848591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.849455 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:39.849467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.850187 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.850192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:39.850397 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.851053 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:39.851070 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:39.853658 (+ 2588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:39.857436 (+ 3778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.857443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.857446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.857841 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.858242 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:39.858248 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:39.858252 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:39.858393 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:39.858940 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:39.858945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:39.859096 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 15:16:39.859263 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 15:16:39.859499 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162}
I20260809 15:16:39.860615 30497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:39.846205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43762 (local address 127.15.11.188:33577)
0809 15:16:39.846363 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:39.846371 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:39.846422 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:16:39.846620 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:39.846627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:39.846692 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:39.846817 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:39.847600 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.848431 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.849597 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:39.850042 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:39.850176 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:39.853904 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:39.853910 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:39.853916 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:39.853951 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:39.857264 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.857961 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.857967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.857971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.858119 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:39.858510 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:39.858517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:39.858520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:39.858816 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:39.858971 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:39.860104 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0809 15:16:39.860371 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 15:16:39.862520 30440 tablet.cc:2369] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:16:39.863564 30268 tablet.cc:2369] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:16:39.865366 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8: Bootstrap starting.
I20260809 15:16:39.869468 30404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:39.870916 30404 log.cc:824] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:39.874358 30404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8: No bootstrap required, opened a new log
I20260809 15:16:39.885499 30404 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } }
I20260809 15:16:39.885953 30404 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:39.886071 30404 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c2f26bb8d9744bc86348e72657b06e8, State: Initialized, Role: FOLLOWER
I20260809 15:16:39.886749 30404 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } }
I20260809 15:16:39.891008 30404 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:16:39.887795 30538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:39.892514 30538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:16:39.912668 30328 master_runner.cc:263] No leader master found from master 9c2f26bb8d9744bc86348e72657b06e8
I20260809 15:16:39.912837 30328 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 15:16:39.915588 30551 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:16:39.915815 30551 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:16:39.948218 30538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:16:39.948575 30538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } }
I20260809 15:16:39.950353 30538 leader_election.cc:288] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f140d1f3ede4380ac9be80ba60f074c (127.15.11.190:37143), 07242d321fdf4011b07f82a2c65b5785 (127.15.11.188:33577)
I20260809 15:16:39.952517 30464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c2f26bb8d9744bc86348e72657b06e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07242d321fdf4011b07f82a2c65b5785" is_pre_election: true
I20260809 15:16:39.953147 30464 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2f26bb8d9744bc86348e72657b06e8 in term 0.
I20260809 15:16:39.953774 30347 leader_election.cc:302] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07242d321fdf4011b07f82a2c65b5785, 9c2f26bb8d9744bc86348e72657b06e8; no voters:
I20260809 15:16:39.956406 30538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:16:39.956563 30538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:16:39.956663 30538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:39.960181 30538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } }
I20260809 15:16:39.960896 30538 leader_election.cc:288] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [CANDIDATE]: Term 1 election: Requested vote from peers 0f140d1f3ede4380ac9be80ba60f074c (127.15.11.190:37143), 07242d321fdf4011b07f82a2c65b5785 (127.15.11.188:33577)
I20260809 15:16:39.952028 30292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c2f26bb8d9744bc86348e72657b06e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f140d1f3ede4380ac9be80ba60f074c" is_pre_election: true
I20260809 15:16:39.961863 30464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c2f26bb8d9744bc86348e72657b06e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07242d321fdf4011b07f82a2c65b5785"
I20260809 15:16:39.961917 30292 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9c2f26bb8d9744bc86348e72657b06e8 in term 0.
I20260809 15:16:39.962067 30464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:39.962354 30291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c2f26bb8d9744bc86348e72657b06e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f140d1f3ede4380ac9be80ba60f074c"
I20260809 15:16:39.962601 30291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:39.966054 30291 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2f26bb8d9744bc86348e72657b06e8 in term 1.
I20260809 15:16:39.966244 30464 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9c2f26bb8d9744bc86348e72657b06e8 in term 1.
I20260809 15:16:39.966601 30347 leader_election.cc:302] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f140d1f3ede4380ac9be80ba60f074c, 9c2f26bb8d9744bc86348e72657b06e8; no voters:
I20260809 15:16:39.967437 30538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:16:39.975765 30538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [term 1 LEADER]: Becoming Leader. State: Replica: 9c2f26bb8d9744bc86348e72657b06e8, State: Running, Role: LEADER
I20260809 15:16:39.976456 30538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } }
I20260809 15:16:39.989396 30553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c2f26bb8d9744bc86348e72657b06e8. Latest consensus state: current_term: 1 leader_uuid: "9c2f26bb8d9744bc86348e72657b06e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:39.989889 30553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:39.994875 30555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:16:39.997284 30555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:16:40.017019 30464 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [term 1 FOLLOWER]: Refusing update from remote peer 9c2f26bb8d9744bc86348e72657b06e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:16:40.018106 30553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:16:40.025602 30291 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [term 1 FOLLOWER]: Refusing update from remote peer 9c2f26bb8d9744bc86348e72657b06e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:16:40.026620 30553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:16:40.065255 30524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c2f26bb8d9744bc86348e72657b06e8. Latest consensus state: current_term: 1 leader_uuid: "9c2f26bb8d9744bc86348e72657b06e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:40.065619 30524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:16:40.069942 30556 mvcc.cc:205] Tried to move back new op lower bound from 7316638105657044992 to 7316638105509376000. Current Snapshot: MvccSnapshot[applied={T|T < 7316638105657044992}]
I20260809 15:16:40.072562 30538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c2f26bb8d9744bc86348e72657b06e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:40.072885 30538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:40.081296 30555 catalog_manager.cc:1347] Generated new cluster ID: 154ff1bfe4504626acd6153fea7ebfb1
I20260809 15:16:40.081399 30555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:16:40.105165 30524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c2f26bb8d9744bc86348e72657b06e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:40.105492 30524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:16:40.111995 30555 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:16:40.114708 30555 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:16:40.130334 30555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8: Generated new TSK 0
I20260809 15:16:40.131426 30555 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:16:40.135336 30509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c2f26bb8d9744bc86348e72657b06e8. Latest consensus state: current_term: 1 leader_uuid: "9c2f26bb8d9744bc86348e72657b06e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:40.135664 30509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:16:40.137678 30553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9c2f26bb8d9744bc86348e72657b06e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:40.137953 30553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c2f26bb8d9744bc86348e72657b06e8 [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:40.140237 30557 mvcc.cc:205] Tried to move back new op lower bound from 7316638105657044992 to 7316638105509376000. Current Snapshot: MvccSnapshot[applied={T|T < 7316638105657044992}]
I20260809 15:16:40.156718 30509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9c2f26bb8d9744bc86348e72657b06e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f140d1f3ede4380ac9be80ba60f074c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37143 } } peers { permanent_uuid: "9c2f26bb8d9744bc86348e72657b06e8" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 37767 } } peers { permanent_uuid: "07242d321fdf4011b07f82a2c65b5785" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 33577 } } }
I20260809 15:16:40.157018 30509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:16:40.743010 30521 catalog_manager.cc:1259] Loaded cluster ID: 154ff1bfe4504626acd6153fea7ebfb1
I20260809 15:16:40.743139 30521 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785: loading cluster ID for follower catalog manager: success
I20260809 15:16:40.745606 30521 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785: acquiring CA information for follower catalog manager: success
I20260809 15:16:40.747251 30521 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 07242d321fdf4011b07f82a2c65b5785: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:16:40.821630 30535 catalog_manager.cc:1259] Loaded cluster ID: 154ff1bfe4504626acd6153fea7ebfb1
I20260809 15:16:40.821774 30535 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c: loading cluster ID for follower catalog manager: success
I20260809 15:16:40.824295 30535 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c: acquiring CA information for follower catalog manager: success
I20260809 15:16:40.825989 30535 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0f140d1f3ede4380ac9be80ba60f074c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:16:41.280607 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 30242
I20260809 15:16:41.281721 30315 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:41.387154 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 30242
I20260809 15:16:41.413065 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 30328
I20260809 15:16:41.414151 30398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:41.518342 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 30328
I20260809 15:16:41.542641 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 30414
I20260809 15:16:41.543889 30489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:41.649950 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 30414
2026-08-09T15:16:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4858 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:16:41 dist-test-slave-8w9v krb5kdc[30575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:16:41 dist-test-slave-8w9v krb5kdc[30575](info): set up 2 sockets
Aug 09 15:16:41 dist-test-slave-8w9v krb5kdc[30575](info): commencing operation
krb5kdc: starting...
W20260809 15:16:42.799590 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:16:42 dist-test-slave-8w9v krb5kdc[30575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:16:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:16:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:16:43.064015 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37315
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:46441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:37315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:43.394426 30592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:43.394796 30592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:43.394948 30592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:43.402004 30592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:16:43.402144 30592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:43.402266 30592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:43.402372 30592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:16:43.402459 30592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:16:43.411916 30592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:46441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:37315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:43.413247 30592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:43.414572 30592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:43.426281 30597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:43.426323 30599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:43.426234 30601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:43.427476 30592 server_base.cc:1034] running on GCE node
I20260809 15:16:43.428341 30592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:43.429733 30592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:43.430969 30592 hybrid_clock.cc:648] HybridClock initialized: now 1786288603430935 us; error 43 us; skew 500 ppm
I20260809 15:16:43.431308 30592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:43 dist-test-slave-8w9v krb5kdc[30575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288603, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:43.434386 30592 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:16:43.435802 30592 webserver.cc:460] Webserver started at http://127.15.11.190:39771/ using document root <none> and password file <none>
I20260809 15:16:43.436578 30592 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:43.436751 30592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:43.437072 30592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:16:43.440026 30592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "11399c87eb6b4e2295c3933838acbf70"
format_stamp: "Formatted at 2026-08-09 15:16:43 on dist-test-slave-8w9v"
server_key: "de7e42430bd17594ac7626b6d202f55c"
server_key_iv: "db774bb3c865a53ae5922909aa62c4b7"
server_key_version: "encryptionkey@0"
I20260809 15:16:43.440871 30592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "11399c87eb6b4e2295c3933838acbf70"
format_stamp: "Formatted at 2026-08-09 15:16:43 on dist-test-slave-8w9v"
server_key: "de7e42430bd17594ac7626b6d202f55c"
server_key_iv: "db774bb3c865a53ae5922909aa62c4b7"
server_key_version: "encryptionkey@0"
I20260809 15:16:43.445753 30592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 15:16:43.448987 30609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:43.450201 30592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:16:43.450383 30592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "11399c87eb6b4e2295c3933838acbf70"
format_stamp: "Formatted at 2026-08-09 15:16:43 on dist-test-slave-8w9v"
server_key: "de7e42430bd17594ac7626b6d202f55c"
server_key_iv: "db774bb3c865a53ae5922909aa62c4b7"
server_key_version: "encryptionkey@0"
I20260809 15:16:43.450584 30592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:43.478608 30592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:43.482401 30592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:43.482679 30592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:43.500909 30592 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:37315
I20260809 15:16:43.500939 30661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:37315 every 8 connection(s)
I20260809 15:16:43.502252 30592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:16:43.506016 30662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:43.507332 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 30592
I20260809 15:16:43.507550 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:16:43.508208 15406 external_mini_cluster.cc:1428] Setting key f454686921fb5fbe865c0c9cf828df76
I20260809 15:16:43.519896 30662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70: Bootstrap starting.
Aug 09 15:16:43 dist-test-slave-8w9v krb5kdc[30575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:43.523777 30662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:43.525041 30662 log.cc:824] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:43.527683 30662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70: No bootstrap required, opened a new log
I20260809 15:16:43.531109 30665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:43.509999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34774 (local address 127.15.11.190:37315)
0809 15:16:43.510906 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:43.510917 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:43.510958 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:16:43.512592 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:43.512613 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:43.512981 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:43.513290 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:43.515073 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:43.516462 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:43.518322 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:43.518904 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:43.519037 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:43.523982 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:43.523990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:43.524003 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:43.524043 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:43.527015 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:43.527701 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:43.527707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:43.527712 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:43.527915 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:43.528338 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:43.528343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:43.528346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:43.528877 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:43.529193 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:43.529694 (+ 501us) server_negotiation.cc:299] Negotiation successful
0809 15:16:43.530128 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":403,"threads_started":1}
I20260809 15:16:43.535121 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:16:43.536295 30662 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11399c87eb6b4e2295c3933838acbf70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37315 } }
I20260809 15:16:43.536645 30662 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:43.536748 30662 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11399c87eb6b4e2295c3933838acbf70, State: Initialized, Role: FOLLOWER
I20260809 15:16:43.537325 30662 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11399c87eb6b4e2295c3933838acbf70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37315 } }
I20260809 15:16:43.537626 30662 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:16:43.537791 30662 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:16:43.537982 30662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:43.540274 30662 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11399c87eb6b4e2295c3933838acbf70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37315 } }
I20260809 15:16:43.540716 30662 leader_election.cc:302] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11399c87eb6b4e2295c3933838acbf70; no voters:
I20260809 15:16:43.541244 30662 leader_election.cc:288] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:16:43.541396 30667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:16:43.542220 30667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [term 1 LEADER]: Becoming Leader. State: Replica: 11399c87eb6b4e2295c3933838acbf70, State: Running, Role: LEADER
I20260809 15:16:43.542789 30667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11399c87eb6b4e2295c3933838acbf70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37315 } }
I20260809 15:16:43.543542 30662 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:16:43.545451 30669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11399c87eb6b4e2295c3933838acbf70. Latest consensus state: current_term: 1 leader_uuid: "11399c87eb6b4e2295c3933838acbf70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11399c87eb6b4e2295c3933838acbf70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37315 } } }
I20260809 15:16:43.545476 30668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11399c87eb6b4e2295c3933838acbf70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11399c87eb6b4e2295c3933838acbf70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37315 } } }
I20260809 15:16:43.545971 30668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:43.546592 30669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70 [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:43.547348 30672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:16:43.552327 30672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:16:43.561497 30672 catalog_manager.cc:1347] Generated new cluster ID: bdd7ce0fc96a42089679f67d031f2370
I20260809 15:16:43.561621 30672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:16:43.575767 30672 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:16:43.577105 30672 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:16:43.592486 30672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11399c87eb6b4e2295c3933838acbf70: Generated new TSK 0
I20260809 15:16:43.593318 30672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:16:43.648000 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 30592
I20260809 15:16:43.649190 30657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:43.745741 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 30592
2026-08-09T15:16:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2093 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:16:43 dist-test-slave-8w9v krb5kdc[30694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:16:43 dist-test-slave-8w9v krb5kdc[30694](info): set up 2 sockets
Aug 09 15:16:43 dist-test-slave-8w9v krb5kdc[30694](info): commencing operation
krb5kdc: starting...
W20260809 15:16:44.891551 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:16:45 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:16:45.151165 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37409
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:37409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:45.479920 30708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:45.480206 30708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:45.480294 30708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:45.487093 30708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:16:45.487187 30708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:45.487231 30708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:45.487286 30708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:16:45.487322 30708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:16:45.496712 30708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:37409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:45.497951 30708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:45.499163 30708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:45.510502 30716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:45.510568 30713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:45.510545 30714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:45.511801 30708 server_base.cc:1034] running on GCE node
I20260809 15:16:45.512593 30708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:45.513922 30708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:45.515169 30708 hybrid_clock.cc:648] HybridClock initialized: now 1786288605515137 us; error 50 us; skew 500 ppm
I20260809 15:16:45.515524 30708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:45 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288605, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:45.518543 30708 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:16:45.519924 30708 webserver.cc:460] Webserver started at http://127.15.11.190:36943/ using document root <none> and password file <none>
I20260809 15:16:45.520623 30708 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:45.520792 30708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:45.521103 30708 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:16:45.524014 30708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "4956f3a6b4464feb9909b55a840c213f"
format_stamp: "Formatted at 2026-08-09 15:16:45 on dist-test-slave-8w9v"
server_key: "d89fda388099937b8c83ce38b675672e"
server_key_iv: "9d1d69c8d4ca4b7be2853b3cc8f4d08f"
server_key_version: "encryptionkey@0"
I20260809 15:16:45.524825 30708 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "4956f3a6b4464feb9909b55a840c213f"
format_stamp: "Formatted at 2026-08-09 15:16:45 on dist-test-slave-8w9v"
server_key: "d89fda388099937b8c83ce38b675672e"
server_key_iv: "9d1d69c8d4ca4b7be2853b3cc8f4d08f"
server_key_version: "encryptionkey@0"
I20260809 15:16:45.529850 30708 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:16:45.533206 30729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:45.534543 30708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:16:45.534725 30708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "4956f3a6b4464feb9909b55a840c213f"
format_stamp: "Formatted at 2026-08-09 15:16:45 on dist-test-slave-8w9v"
server_key: "d89fda388099937b8c83ce38b675672e"
server_key_iv: "9d1d69c8d4ca4b7be2853b3cc8f4d08f"
server_key_version: "encryptionkey@0"
I20260809 15:16:45.534937 30708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:45.549732 30708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:45.553478 30708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:45.553748 30708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:45.571812 30708 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:37409
I20260809 15:16:45.571838 30784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:37409 every 8 connection(s)
I20260809 15:16:45.573287 30708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:16:45.574846 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 30708
I20260809 15:16:45.575088 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:16:45.575775 15406 external_mini_cluster.cc:1428] Setting key f2b5f012aab3b951a6a9e4129c5f4d04
I20260809 15:16:45.577497 30785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:16:45 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:45.591881 30785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Bootstrap starting.
I20260809 15:16:45.595363 30785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:45.596856 30785 log.cc:824] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:45.600381 30785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: No bootstrap required, opened a new log
I20260809 15:16:45.600714 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:45.577921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.15.11.190:37409)
0809 15:16:45.578793 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:45.578808 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:45.578856 (+ 48us) server_negotiation.cc:407] Connection header received
0809 15:16:45.580902 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:45.580929 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:45.581423 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:45.581844 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:45.583210 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:45.585310 (+ 2100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:45.586813 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:45.588056 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:45.588286 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:45.592933 (+ 4647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:45.592940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:45.592953 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:45.592987 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:45.596056 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:45.597047 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:45.597058 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:45.597064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:45.597223 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:45.597704 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:45.597711 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:45.597715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:45.598437 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:45.598659 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:45.599249 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 15:16:45.599593 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":376,"threads_started":1}
I20260809 15:16:45.608556 30785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:45.609005 30785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:45.609130 30785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4956f3a6b4464feb9909b55a840c213f, State: Initialized, Role: FOLLOWER
I20260809 15:16:45.609814 30785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:45.610164 30785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:16:45.610344 30785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:16:45.610500 30785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:45.613087 30785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:45.613752 30785 leader_election.cc:302] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4956f3a6b4464feb9909b55a840c213f; no voters:
I20260809 15:16:45.614341 30785 leader_election.cc:288] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:16:45.614490 30790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:16:45.615274 30790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 1 LEADER]: Becoming Leader. State: Replica: 4956f3a6b4464feb9909b55a840c213f, State: Running, Role: LEADER
I20260809 15:16:45.615849 30790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:45.616750 30785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:16:45.618978 30792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4956f3a6b4464feb9909b55a840c213f. Latest consensus state: current_term: 1 leader_uuid: "4956f3a6b4464feb9909b55a840c213f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } } }
I20260809 15:16:45.619463 30792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:45.619378 30791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4956f3a6b4464feb9909b55a840c213f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } } }
I20260809 15:16:45.619705 30791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:45.623437 30795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:16:45.628314 30795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:16:45.637022 30795 catalog_manager.cc:1347] Generated new cluster ID: 09c6a08142fc4dfa84b1c9b17d36d7d5
I20260809 15:16:45.637142 30795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:16:45.678793 30795 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:16:45.680255 30795 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:16:45.687004 30795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Generated new TSK 0
I20260809 15:16:45.687819 30795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:16:45.759624 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37409
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:46.088686 30813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:46.088976 30813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:46.089150 30813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:46.095963 30813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:46.096093 30813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:46.096273 30813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:16:46.106066 30813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:46.107368 30813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:46.108857 30813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:46.122956 30818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:46.122956 30821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:46.122949 30819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:46.123687 30813 server_base.cc:1034] running on GCE node
I20260809 15:16:46.124310 30813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:46.125097 30813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:46.126281 30813 hybrid_clock.cc:648] HybridClock initialized: now 1786288606126228 us; error 48 us; skew 500 ppm
I20260809 15:16:46.126573 30813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:46 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288606, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:46.129730 30813 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:16:46.131253 30813 webserver.cc:460] Webserver started at http://127.15.11.129:35785/ using document root <none> and password file <none>
I20260809 15:16:46.131912 30813 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:46.132057 30813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:46.132370 30813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:16:46.135243 30813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "8eae1d6716974792a2501bb3f006762a"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "6ad38e81dbd6729872a6200a28cd5c2b"
server_key_iv: "4dc43bef3d25b13a93bbfe984c3bcee8"
server_key_version: "encryptionkey@0"
I20260809 15:16:46.136049 30813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "8eae1d6716974792a2501bb3f006762a"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "6ad38e81dbd6729872a6200a28cd5c2b"
server_key_iv: "4dc43bef3d25b13a93bbfe984c3bcee8"
server_key_version: "encryptionkey@0"
I20260809 15:16:46.141074 30813 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 15:16:46.144320 30828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:46.145560 30813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:16:46.145723 30813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "8eae1d6716974792a2501bb3f006762a"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "6ad38e81dbd6729872a6200a28cd5c2b"
server_key_iv: "4dc43bef3d25b13a93bbfe984c3bcee8"
server_key_version: "encryptionkey@0"
I20260809 15:16:46.145928 30813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:46.174695 30813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:46.178387 30813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:46.178684 30813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:46.179592 30813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:46.180876 30813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:46.181005 30813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:46.181154 30813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:46.181234 30813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:46.214185 30813 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:35641
I20260809 15:16:46.214231 30956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:35641 every 8 connection(s)
I20260809 15:16:46.215468 30813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:16:46.225143 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 30813
I20260809 15:16:46.225375 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:16:46.226096 15406 external_mini_cluster.cc:1428] Setting key 40f9a4abf1fc58b2588c0a2002e77601
Aug 09 15:16:46 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288606, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:46.236826 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:46.217952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:54139 (local address 127.15.11.190:37409)
0809 15:16:46.218421 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:46.218426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:46.219315 (+ 889us) server_negotiation.cc:407] Connection header received
0809 15:16:46.221125 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:46.221130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:46.221229 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:46.221348 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:46.222850 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:46.223607 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:46.225597 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:46.225980 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:46.226083 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:46.230043 (+ 3960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:46.230052 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:46.230059 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:46.230106 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:46.233568 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:46.234284 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:46.234292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:46.234298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:46.234454 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:46.234856 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:46.234863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:46.234869 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:46.235254 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:46.235414 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:46.236097 (+ 683us) server_negotiation.cc:299] Negotiation successful
0809 15:16:46.236371 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":253,"threads_started":1}
I20260809 15:16:46.237429 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:46.218540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:54139)
0809 15:16:46.219175 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:46.219210 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:46.220853 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:46.221583 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:46.221591 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:46.222092 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:46.222637 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:46.222653 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:46.223743 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:46.223748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:46.225417 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:46.225433 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:46.226217 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:46.226225 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:46.226525 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:46.227320 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:46.227348 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:46.229823 (+ 2475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:46.233705 (+ 3882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:46.233711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:46.233739 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:46.234148 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:46.234562 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:46.234565 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:46.234567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:46.234739 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:46.235375 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:46.235380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:46.235694 (+ 314us) client_negotiation.cc:769] Sending connection context
0809 15:16:46.235924 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 15:16:46.236367 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":265,"threads_started":1}
I20260809 15:16:46.239086 30957 heartbeater.cc:347] Connected to a master server at 127.15.11.190:37409
I20260809 15:16:46.239423 30957 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:46.240281 30957 heartbeater.cc:511] Master 127.15.11.190:37409 requested a full tablet report, sending...
I20260809 15:16:46.242588 30747 ts_manager.cc:194] Registered new tserver with Master: 8eae1d6716974792a2501bb3f006762a (127.15.11.129:35641)
I20260809 15:16:46.244761 30747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:54139
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:16:46.287791 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37409
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:46.610038 30963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:46.610327 30963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:46.610430 30963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:46.617630 30963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:46.617725 30963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:46.617884 30963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:16:46.627164 30963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:46.628278 30963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:46.629583 30963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:46.643026 30969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:46.643143 30968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:46.643132 30971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:46.643797 30963 server_base.cc:1034] running on GCE node
I20260809 15:16:46.644551 30963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:46.645473 30963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:46.646693 30963 hybrid_clock.cc:648] HybridClock initialized: now 1786288606646635 us; error 53 us; skew 500 ppm
I20260809 15:16:46.647006 30963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:46 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288606, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:46.650365 30963 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:16:46.651722 30963 webserver.cc:460] Webserver started at http://127.15.11.130:45549/ using document root <none> and password file <none>
I20260809 15:16:46.652503 30963 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:46.652612 30963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:46.652858 30963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:16:46.655673 30963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "bbf475bd1f5e593f144f32fd52df183a"
server_key_iv: "2df77812a8f82acdc0739b5cb0e45190"
server_key_version: "encryptionkey@0"
I20260809 15:16:46.656498 30963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "bbf475bd1f5e593f144f32fd52df183a"
server_key_iv: "2df77812a8f82acdc0739b5cb0e45190"
server_key_version: "encryptionkey@0"
I20260809 15:16:46.661473 30963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:16:46.664804 30979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:46.666076 30963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:16:46.666249 30963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "bbf475bd1f5e593f144f32fd52df183a"
server_key_iv: "2df77812a8f82acdc0739b5cb0e45190"
server_key_version: "encryptionkey@0"
I20260809 15:16:46.666450 30963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:46.677546 30963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:46.681188 30963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:46.681485 30963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:46.682487 30963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:46.683743 30963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:46.683844 30963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:46.683988 30963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:46.684064 30963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:46.716462 30963 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:38913
I20260809 15:16:46.716506 31102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:38913 every 8 connection(s)
I20260809 15:16:46.717947 30963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:16:46.722760 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 30963
I20260809 15:16:46.722956 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:16:46.723438 15406 external_mini_cluster.cc:1428] Setting key 91de5f97357473153e6518d778f53210
Aug 09 15:16:46 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288606, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:46.739096 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:46.720510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:40159 (local address 127.15.11.190:37409)
0809 15:16:46.720694 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:46.720698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:46.721842 (+ 1144us) server_negotiation.cc:407] Connection header received
0809 15:16:46.723751 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:46.723760 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:46.723850 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:46.723972 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:46.725775 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:46.726480 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:46.728023 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:46.728526 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:46.728660 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:46.732417 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:46.732426 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:46.732435 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:46.732479 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:46.735955 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:46.736703 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:46.736710 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:46.736713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:46.736845 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:46.737299 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:46.737310 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:46.737312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:46.737576 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:46.737734 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:46.738530 (+ 796us) server_negotiation.cc:299] Negotiation successful
0809 15:16:46.738754 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:16:46.740065 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:46.720985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:40159)
0809 15:16:46.721679 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:46.721720 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:46.723445 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:46.724296 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:46.724304 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:46.724942 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:46.725608 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:46.725627 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:46.726640 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:46.726652 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:46.727875 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:46.727887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:46.728747 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:46.728755 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:46.729016 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:46.729726 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:46.729748 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:46.732197 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:46.736156 (+ 3959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:46.736171 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:46.736182 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:46.736580 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:46.736968 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:46.736973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:46.736978 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:46.737172 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:46.737709 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:46.737716 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:46.738139 (+ 423us) client_negotiation.cc:769] Sending connection context
0809 15:16:46.738450 (+ 311us) client_negotiation.cc:240] Negotiation successful
0809 15:16:46.738940 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":267,"threads_started":1}
I20260809 15:16:46.741592 31103 heartbeater.cc:347] Connected to a master server at 127.15.11.190:37409
I20260809 15:16:46.741892 31103 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:46.742574 31103 heartbeater.cc:511] Master 127.15.11.190:37409 requested a full tablet report, sending...
I20260809 15:16:46.743983 30747 ts_manager.cc:194] Registered new tserver with Master: e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913)
I20260809 15:16:46.744786 30747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:40159
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:16:46.783093 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37409
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:47.110505 31109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:47.110771 31109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:47.110947 31109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:47.119576 31109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:47.119702 31109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:47.119863 31109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:16:47.129280 31109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:37409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:47.130856 31109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:47.132710 31109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:47.147516 31114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:47.147578 31115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:47.147636 31118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:47.148505 31109 server_base.cc:1034] running on GCE node
I20260809 15:16:47.149091 31109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:47.150048 31109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:47.151314 31109 hybrid_clock.cc:648] HybridClock initialized: now 1786288607151261 us; error 46 us; skew 500 ppm
I20260809 15:16:47.151655 31109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:47 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288607, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:47.154629 31109 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:16:47.156049 31109 webserver.cc:460] Webserver started at http://127.15.11.131:39791/ using document root <none> and password file <none>
I20260809 15:16:47.156766 31109 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:47.156870 31109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:47.157124 31109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:16:47.159884 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "917ff1ff919f4e879e22e1cb43ca84c4"
format_stamp: "Formatted at 2026-08-09 15:16:47 on dist-test-slave-8w9v"
server_key: "efebdd8fc2ffe180fb4dca0567af819b"
server_key_iv: "bfee33e38836a5562f8d9ff11b03195c"
server_key_version: "encryptionkey@0"
I20260809 15:16:47.160667 31109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "917ff1ff919f4e879e22e1cb43ca84c4"
format_stamp: "Formatted at 2026-08-09 15:16:47 on dist-test-slave-8w9v"
server_key: "efebdd8fc2ffe180fb4dca0567af819b"
server_key_iv: "bfee33e38836a5562f8d9ff11b03195c"
server_key_version: "encryptionkey@0"
I20260809 15:16:47.165613 31109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:16:47.168766 31127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:47.170091 31109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:16:47.170264 31109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "917ff1ff919f4e879e22e1cb43ca84c4"
format_stamp: "Formatted at 2026-08-09 15:16:47 on dist-test-slave-8w9v"
server_key: "efebdd8fc2ffe180fb4dca0567af819b"
server_key_iv: "bfee33e38836a5562f8d9ff11b03195c"
server_key_version: "encryptionkey@0"
I20260809 15:16:47.170509 31109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:47.205819 31109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:47.209755 31109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:47.210062 31109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:47.211104 31109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:47.212529 31109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:47.212641 31109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:47.212787 31109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:47.212859 31109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:47.247641 31109 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:45043
I20260809 15:16:47.247730 31253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:45043 every 8 connection(s)
I20260809 15:16:47.248538 30957 heartbeater.cc:503] Master 127.15.11.190:37409 was elected leader, sending a full tablet report...
I20260809 15:16:47.249097 31109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:16:47.257215 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 31109
I20260809 15:16:47.257404 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:16:47.257838 15406 external_mini_cluster.cc:1428] Setting key c5c1f7a5e8d5cbaad167e02f4d85abb1
Aug 09 15:16:47 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288607, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:47.269129 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.251909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:45931 (local address 127.15.11.190:37409)
0809 15:16:47.252405 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.252410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.253339 (+ 929us) server_negotiation.cc:407] Connection header received
0809 15:16:47.255181 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.255189 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.255279 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.255435 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:47.256987 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.257926 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.259861 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.260527 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.260693 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.263864 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:47.263870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:47.263874 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:47.263905 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:47.266585 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.267196 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.267199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.267201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.267318 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.267681 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.267685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.267686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.267931 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:47.268039 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.268713 (+ 674us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.268874 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":280,"threads_started":1}
I20260809 15:16:47.270161 31260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.252512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:45931)
0809 15:16:47.253189 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:47.253234 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:47.254925 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:47.255671 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:47.255679 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:47.256266 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:47.256809 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.256825 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.258059 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.258064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.259539 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.259553 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.260760 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.260766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.260968 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.261575 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:47.261597 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:47.263677 (+ 2080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:47.266702 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:47.266708 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:47.266719 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:47.267100 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:47.267421 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:47.267424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:47.267426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:47.267592 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:47.268032 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:47.268038 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:47.268396 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 15:16:47.268619 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 15:16:47.268977 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":272,"threads_started":1}
I20260809 15:16:47.271703 31254 heartbeater.cc:347] Connected to a master server at 127.15.11.190:37409
I20260809 15:16:47.272020 31254 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:47.272943 31254 heartbeater.cc:511] Master 127.15.11.190:37409 requested a full tablet report, sending...
I20260809 15:16:47.274448 30747 ts_manager.cc:194] Registered new tserver with Master: 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
I20260809 15:16:47.275254 30747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:45931
I20260809 15:16:47.285439 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:16:47.301893 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.290014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35050 (local address 127.15.11.190:37409)
0809 15:16:47.290254 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.290261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.290537 (+ 276us) server_negotiation.cc:407] Connection header received
0809 15:16:47.290667 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.290671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.290734 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.290813 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:47.292170 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.292890 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.294238 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.294640 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.294741 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.296930 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:47.296934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:47.296937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:47.296967 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:47.299544 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.300271 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.300275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.300277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.300420 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.300705 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.300707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.300709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.301009 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:47.301189 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.301438 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.301597 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 15:16:47.305939 30747 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:16:47.312664 30747 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:16:47.336665 31272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.325165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.15.11.130:38913)
0809 15:16:47.327097 (+ 1932us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.327108 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.327135 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:16:47.327233 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.327240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.327510 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.328616 (+ 1106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:47.328913 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.330944 (+ 2031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.334141 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.335455 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.335603 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.335795 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.336158 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.336328 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1782,"thread_start_us":1644,"threads_started":1}
I20260809 15:16:47.336701 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.324930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:38913 (local address 127.0.0.1:34970)
0809 15:16:47.325830 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:47.325873 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:47.326035 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:47.327773 (+ 1738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:47.327784 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:47.327828 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:47.328356 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.328369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.331519 (+ 3150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.331526 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.333950 (+ 2424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.333966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.335671 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.335677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.335853 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.335928 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 15:16:47.336077 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 15:16:47.336316 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":5802,"threads_started":1}
I20260809 15:16:47.339504 31270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.325194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:45043 (local address 127.0.0.1:49514)
0809 15:16:47.331037 (+ 5843us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:47.331055 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:47.331179 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:47.332115 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:47.332120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:47.332162 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:47.332651 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.332663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.334411 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.334415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.336663 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.336677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.338718 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.338723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.338902 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.338931 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 15:16:47.339033 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 15:16:47.339154 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5718,"thread_start_us":276,"threads_started":1}
I20260809 15:16:47.339715 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.325540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49514 (local address 127.15.11.131:45043)
0809 15:16:47.326683 (+ 1143us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.326691 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.331180 (+ 4489us) server_negotiation.cc:407] Connection header received
0809 15:16:47.331385 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.331395 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.331647 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.331798 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:47.332875 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.334252 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.336851 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.338211 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.338313 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.338537 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.339132 (+ 595us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.339303 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":783,"threads_started":1}
I20260809 15:16:47.340485 31275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.326743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.0.0.1:57380)
0809 15:16:47.331661 (+ 4918us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:47.331679 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:47.331883 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:47.332737 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:47.332741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:47.332760 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:47.333273 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.333289 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.335197 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.335202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.337296 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.337311 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.339643 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.339650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.339800 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.339841 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 15:16:47.339983 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 15:16:47.340113 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4784,"thread_start_us":4611,"threads_started":1}
I20260809 15:16:47.340517 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.327216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57380 (local address 127.15.11.129:35641)
0809 15:16:47.327911 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.327921 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.331781 (+ 3860us) server_negotiation.cc:407] Connection header received
0809 15:16:47.332103 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.332109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.332328 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.332472 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:47.333486 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.335043 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.337609 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.339127 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.339225 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.339451 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.340032 (+ 581us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.340207 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":311,"threads_started":1}
I20260809 15:16:47.341166 31029 tablet_service.cc:1467] Processing CreateTablet for tablet 6cfbc82d062d47f88394c1dda4755687 (DEFAULT_TABLE table=test-table [id=ae8ee5cab4824d238d0e2cc63da53eae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:16:47.342770 31029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cfbc82d062d47f88394c1dda4755687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:47.343811 31181 tablet_service.cc:1467] Processing CreateTablet for tablet 6cfbc82d062d47f88394c1dda4755687 (DEFAULT_TABLE table=test-table [id=ae8ee5cab4824d238d0e2cc63da53eae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:16:47.344033 30884 tablet_service.cc:1467] Processing CreateTablet for tablet 6cfbc82d062d47f88394c1dda4755687 (DEFAULT_TABLE table=test-table [id=ae8ee5cab4824d238d0e2cc63da53eae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:16:47.345136 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cfbc82d062d47f88394c1dda4755687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:47.345240 31181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cfbc82d062d47f88394c1dda4755687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:47.356802 31277 tablet_bootstrap.cc:492] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: Bootstrap starting.
I20260809 15:16:47.358668 31278 tablet_bootstrap.cc:492] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Bootstrap starting.
I20260809 15:16:47.360304 31279 tablet_bootstrap.cc:492] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: Bootstrap starting.
I20260809 15:16:47.362038 31277 tablet_bootstrap.cc:654] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:47.362258 31278 tablet_bootstrap.cc:654] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:47.363131 31279 tablet_bootstrap.cc:654] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:47.363329 31277 log.cc:824] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:47.363328 31278 log.cc:824] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:47.364365 31279 log.cc:824] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:47.366602 31277 tablet_bootstrap.cc:492] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: No bootstrap required, opened a new log
I20260809 15:16:47.366608 31278 tablet_bootstrap.cc:492] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: No bootstrap required, opened a new log
I20260809 15:16:47.366858 31279 tablet_bootstrap.cc:492] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: No bootstrap required, opened a new log
I20260809 15:16:47.366876 31277 ts_tablet_manager.cc:1397] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260809 15:16:47.366871 31278 ts_tablet_manager.cc:1397] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 15:16:47.367127 31279 ts_tablet_manager.cc:1397] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 15:16:47.374898 31279 raft_consensus.cc:348] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } }
I20260809 15:16:47.375325 31279 raft_consensus.cc:374] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:47.375448 31279 raft_consensus.cc:729] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917ff1ff919f4e879e22e1cb43ca84c4, State: Initialized, Role: FOLLOWER
I20260809 15:16:47.376052 31279 consensus_queue.cc:260] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } }
I20260809 15:16:47.377213 31254 heartbeater.cc:503] Master 127.15.11.190:37409 was elected leader, sending a full tablet report...
I20260809 15:16:47.377467 31277 raft_consensus.cc:348] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } }
I20260809 15:16:47.377920 31277 raft_consensus.cc:374] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:47.377722 31278 raft_consensus.cc:348] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } }
I20260809 15:16:47.378032 31277 raft_consensus.cc:729] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eae1d6716974792a2501bb3f006762a, State: Initialized, Role: FOLLOWER
I20260809 15:16:47.378139 31278 raft_consensus.cc:374] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:47.378232 31278 raft_consensus.cc:729] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c9ee8ebc6e4970a1bd6907bdef3b9d, State: Initialized, Role: FOLLOWER
I20260809 15:16:47.378645 31277 consensus_queue.cc:260] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } }
I20260809 15:16:47.378777 31278 consensus_queue.cc:260] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } }
I20260809 15:16:47.379673 31279 ts_tablet_manager.cc:1428] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260809 15:16:47.382148 31277 ts_tablet_manager.cc:1428] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: Time spent starting tablet: real 0.015s user 0.014s sys 0.002s
I20260809 15:16:47.383848 31103 heartbeater.cc:503] Master 127.15.11.190:37409 was elected leader, sending a full tablet report...
I20260809 15:16:47.385303 31278 ts_tablet_manager.cc:1428] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Time spent starting tablet: real 0.018s user 0.006s sys 0.013s
I20260809 15:16:47.456149 31285 raft_consensus.cc:482] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:16:47.456485 31285 raft_consensus.cc:504] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } }
I20260809 15:16:47.458784 31285 leader_election.cc:288] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913), 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
I20260809 15:16:47.466180 31272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.459202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:56281 (local address 127.15.11.130:38913)
0809 15:16:47.459452 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.459460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.460433 (+ 973us) server_negotiation.cc:407] Connection header received
0809 15:16:47.460630 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.460635 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.460714 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.460808 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:47.461530 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.462421 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.464341 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.465155 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.465269 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.465364 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.465789 (+ 425us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.465921 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 15:16:47.466192 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.459355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:60505 (local address 127.15.11.131:45043)
0809 15:16:47.459608 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.459616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.460533 (+ 917us) server_negotiation.cc:407] Connection header received
0809 15:16:47.460725 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.460734 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.460825 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.460931 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:47.461670 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.462610 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.464455 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.465231 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.465397 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.465472 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.465827 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.465955 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 15:16:47.466326 31288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.459231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:38913 (local address 127.15.11.129:56281)
0809 15:16:47.460304 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:47.460318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:47.460464 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:47.460840 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:47.460845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:47.460875 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:47.461318 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.461354 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.462551 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.462559 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.464188 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.464200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.465387 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.465393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.465588 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.465641 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 15:16:47.465725 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 15:16:47.465865 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"mutex_wait_us":71,"thread_start_us":527,"threads_started":1}
I20260809 15:16:47.466351 31289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.459267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:45043 (local address 127.15.11.129:60505)
0809 15:16:47.460406 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:47.460432 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:47.460575 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:47.460974 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:47.460977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:47.461011 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:47.461375 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.461382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.462756 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.462759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.464266 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:47.464284 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.465477 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.465481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:47.465639 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.465674 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 15:16:47.465778 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:16:47.465909 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":23,"spinlock_wait_cycles":48384,"thread_start_us":695,"threads_started":1}
I20260809 15:16:47.467435 31053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cfbc82d062d47f88394c1dda4755687" candidate_uuid: "8eae1d6716974792a2501bb3f006762a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" is_pre_election: true
I20260809 15:16:47.467415 31201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cfbc82d062d47f88394c1dda4755687" candidate_uuid: "8eae1d6716974792a2501bb3f006762a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" is_pre_election: true
W20260809 15:16:47.467963 30958 tablet.cc:2369] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:16:47.467932 31053 raft_consensus.cc:2454] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eae1d6716974792a2501bb3f006762a in term 0.
I20260809 15:16:47.467963 31201 raft_consensus.cc:2454] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eae1d6716974792a2501bb3f006762a in term 0.
I20260809 15:16:47.468668 30834 leader_election.cc:302] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eae1d6716974792a2501bb3f006762a, e7c9ee8ebc6e4970a1bd6907bdef3b9d; no voters:
I20260809 15:16:47.469076 31285 raft_consensus.cc:2781] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:16:47.469228 31285 raft_consensus.cc:482] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:16:47.469340 31285 raft_consensus.cc:3037] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:16:47.469856 31104 tablet.cc:2369] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:16:47.471971 31285 raft_consensus.cc:504] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } }
I20260809 15:16:47.472643 31285 leader_election.cc:288] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 1 election: Requested vote from peers e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913), 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
I20260809 15:16:47.473115 31053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cfbc82d062d47f88394c1dda4755687" candidate_uuid: "8eae1d6716974792a2501bb3f006762a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d"
I20260809 15:16:47.473313 31053 raft_consensus.cc:3037] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:47.473245 31201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cfbc82d062d47f88394c1dda4755687" candidate_uuid: "8eae1d6716974792a2501bb3f006762a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917ff1ff919f4e879e22e1cb43ca84c4"
I20260809 15:16:47.473537 31201 raft_consensus.cc:3037] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:47.475739 31053 raft_consensus.cc:2454] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eae1d6716974792a2501bb3f006762a in term 1.
I20260809 15:16:47.476114 30834 leader_election.cc:302] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eae1d6716974792a2501bb3f006762a, e7c9ee8ebc6e4970a1bd6907bdef3b9d; no voters:
I20260809 15:16:47.476256 31201 raft_consensus.cc:2454] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eae1d6716974792a2501bb3f006762a in term 1.
I20260809 15:16:47.476425 31285 raft_consensus.cc:2781] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:16:47.476970 31285 raft_consensus.cc:686] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 1 LEADER]: Becoming Leader. State: Replica: 8eae1d6716974792a2501bb3f006762a, State: Running, Role: LEADER
I20260809 15:16:47.477465 31285 consensus_queue.cc:237] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } }
I20260809 15:16:47.482143 30745 catalog_manager.cc:5495] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a reported cstate change: term changed from 0 to 1, leader changed from <none> to 8eae1d6716974792a2501bb3f006762a (127.15.11.129). New cstate: current_term: 1 leader_uuid: "8eae1d6716974792a2501bb3f006762a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } health_report { overall_health: HEALTHY } } }
I20260809 15:16:47.499466 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.493935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57386 (local address 127.15.11.129:35641)
0809 15:16:47.494139 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.494144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.494163 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:16:47.494403 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.494407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.494490 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.494636 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:16:47.495017 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.495768 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.497341 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.497771 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.497909 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.498410 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:16:47.498865 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:16:47.499020 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.499118 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.499195 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 15:16:47.501333 31256 tablet.cc:2369] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:16:47.507488 31201 raft_consensus.cc:1261] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [term 1 FOLLOWER]: Refusing update from remote peer 8eae1d6716974792a2501bb3f006762a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:16:47.507561 31053 raft_consensus.cc:1261] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 1 FOLLOWER]: Refusing update from remote peer 8eae1d6716974792a2501bb3f006762a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:16:47.508544 31290 consensus_queue.cc:1035] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:16:47.508877 31285 consensus_queue.cc:1035] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [LEADER]: Connected to new peer: Peer: permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:16:47.520025 31293 mvcc.cc:205] Tried to move back new op lower bound from 7316638136341417984 to 7316638136230928384. Current Snapshot: MvccSnapshot[applied={T|T < 7316638136341417984}]
I20260809 15:16:47.524240 31294 mvcc.cc:205] Tried to move back new op lower bound from 7316638136341417984 to 7316638136230928384. Current Snapshot: MvccSnapshot[applied={T|T < 7316638136341417984}]
I20260809 15:16:47.537252 30743 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:16:47.537563 30743 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:16:47.541405 30743 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Sending DeleteTablet for 3 replicas of tablet 6cfbc82d062d47f88394c1dda4755687
I20260809 15:16:47.542562 31029 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cfbc82d062d47f88394c1dda4755687 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:16:47 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:34970
I20260809 15:16:47.542587 31181 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cfbc82d062d47f88394c1dda4755687 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:16:47 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:49514
I20260809 15:16:47.543057 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 30813
I20260809 15:16:47.543639 30884 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cfbc82d062d47f88394c1dda4755687 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:16:47 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:57380
I20260809 15:16:47.554150 31305 tablet_replica.cc:330] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: stopping tablet replica
I20260809 15:16:47.555441 31305 raft_consensus.cc:2229] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:16:47.557109 31305 raft_consensus.cc:2258] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:16:47.560904 31303 tablet_replica.cc:330] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: stopping tablet replica
I20260809 15:16:47.561477 31305 ts_tablet_manager.cc:1905] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:16:47.561497 31304 tablet_replica.cc:330] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: stopping tablet replica
I20260809 15:16:47.562067 31304 raft_consensus.cc:2229] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:16:47.562570 31303 raft_consensus.cc:2229] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:16:47.563110 31303 raft_consensus.cc:2258] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:16:47.563583 31304 raft_consensus.cc:2258] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:16:47.565833 31303 ts_tablet_manager.cc:1905] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:16:47.570003 31305 ts_tablet_manager.cc:1918] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:16:47.570055 31304 ts_tablet_manager.cc:1905] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:16:47.570225 31305 log.cc:1196] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/6cfbc82d062d47f88394c1dda4755687
I20260809 15:16:47.570953 31305 ts_tablet_manager.cc:1939] T 6cfbc82d062d47f88394c1dda4755687 P 8eae1d6716974792a2501bb3f006762a: Deleting consensus metadata
Aug 09 15:16:47 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:16:47.572643 31303 ts_tablet_manager.cc:1918] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:16:47.572858 31303 log.cc:1196] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/6cfbc82d062d47f88394c1dda4755687
I20260809 15:16:47.573320 30731 catalog_manager.cc:4841] TS 8eae1d6716974792a2501bb3f006762a (127.15.11.129:35641): tablet 6cfbc82d062d47f88394c1dda4755687 (table test-table [id=ae8ee5cab4824d238d0e2cc63da53eae]) successfully deleted
I20260809 15:16:47.573462 31303 ts_tablet_manager.cc:1939] T 6cfbc82d062d47f88394c1dda4755687 P 917ff1ff919f4e879e22e1cb43ca84c4: Deleting consensus metadata
I20260809 15:16:47.574976 30730 catalog_manager.cc:4841] TS 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043): tablet 6cfbc82d062d47f88394c1dda4755687 (table test-table [id=ae8ee5cab4824d238d0e2cc63da53eae]) successfully deleted
I20260809 15:16:47.580464 31304 ts_tablet_manager.cc:1918] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:16:47.580670 31304 log.cc:1196] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/6cfbc82d062d47f88394c1dda4755687
I20260809 15:16:47.581099 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.544587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.15.11.129:35641)
0809 15:16:47.544811 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.544820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.561017 (+ 16197us) server_negotiation.cc:407] Connection header received
0809 15:16:47.561348 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.561357 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.561442 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.561570 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:47.563559 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.564743 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.566631 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.567395 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.567495 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.573429 (+ 5934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:47.573444 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:47.573463 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:47.573510 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:47.577082 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.578070 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.578079 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.578083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.578259 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.578740 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.578745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.578748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.579194 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:47.579474 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.580127 (+ 653us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.580625 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:16:47.581490 31304 ts_tablet_manager.cc:1939] T 6cfbc82d062d47f88394c1dda4755687 P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Deleting consensus metadata
I20260809 15:16:47.582589 30952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:47.584308 30733 catalog_manager.cc:4841] TS e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913): tablet 6cfbc82d062d47f88394c1dda4755687 (table test-table [id=ae8ee5cab4824d238d0e2cc63da53eae]) successfully deleted
I20260809 15:16:47.680182 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 30813
I20260809 15:16:47.703104 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 30963
Aug 09 15:16:47 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:16:47.724993 31272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.703982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34978 (local address 127.15.11.130:38913)
0809 15:16:47.704183 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.704188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.704357 (+ 169us) server_negotiation.cc:407] Connection header received
0809 15:16:47.704544 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.704550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.704643 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.704747 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:47.706029 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.706825 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.708151 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.708568 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.708709 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.717965 (+ 9256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:47.717980 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:47.717998 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:47.718049 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:47.721848 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.722561 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.722570 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.722574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.722743 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.723100 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.723105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.723109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.723611 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:47.723881 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.724345 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.724591 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:16:47.725685 31098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:47.819962 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 30963
I20260809 15:16:47.841315 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 31109
Aug 09 15:16:47 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:16:47.856820 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:47.842099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49526 (local address 127.15.11.131:45043)
0809 15:16:47.842289 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:47.842297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:47.842348 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:16:47.842582 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:47.842587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:47.842654 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:47.842758 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:47.844016 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.844828 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.846170 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:47.846602 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:47.846732 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:47.851134 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:47.851150 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:47.851166 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:47.851198 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:47.854016 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.854692 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.854700 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.854702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.854850 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:47.855236 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:47.855242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:47.855245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:47.855601 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:47.855928 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:47.856273 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 15:16:47.856498 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:16:47.857498 31247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:47.947566 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 31109
I20260809 15:16:47.968860 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 30708
I20260809 15:16:47.970080 30779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:48.081944 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 30708
I20260809 15:16:48.108412 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37409
--webserver_interface=127.15.11.190
--webserver_port=36943
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:37409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:48.445911 31311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:48.446184 31311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:48.446349 31311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:48.453191 31311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:16:48.453281 31311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:48.453397 31311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:48.453490 31311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:16:48.453572 31311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:16:48.463094 31311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:37409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=36943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:48.464314 31311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:48.465670 31311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:48.490613 31321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:48.490613 31319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:48.490867 31318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:48.491264 31311 server_base.cc:1034] running on GCE node
I20260809 15:16:48.492254 31311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:48.493810 31311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:48.495050 31311 hybrid_clock.cc:648] HybridClock initialized: now 1786288608495005 us; error 55 us; skew 500 ppm
I20260809 15:16:48.495458 31311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:48 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288608, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:48.498992 31311 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:16:48.500705 31311 webserver.cc:460] Webserver started at http://127.15.11.190:36943/ using document root <none> and password file <none>
I20260809 15:16:48.501461 31311 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:48.501631 31311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:48.507660 31311 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 15:16:48.510605 31330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:48.511870 31311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:16:48.512068 31311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "4956f3a6b4464feb9909b55a840c213f"
format_stamp: "Formatted at 2026-08-09 15:16:45 on dist-test-slave-8w9v"
server_key: "d89fda388099937b8c83ce38b675672e"
server_key_iv: "9d1d69c8d4ca4b7be2853b3cc8f4d08f"
server_key_version: "encryptionkey@0"
I20260809 15:16:48.513207 31311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:48.552770 31311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:48.556983 31311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:48.557274 31311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:48.576601 31311 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:37409
I20260809 15:16:48.576622 31391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:37409 every 8 connection(s)
I20260809 15:16:48.578200 31311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:16:48.585338 31392 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:16:48.585949 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 31311
I20260809 15:16:48.587013 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:35641
--local_ip_for_outbound_sockets=127.15.11.129
--tserver_master_addrs=127.15.11.190:37409
--webserver_port=35785
--webserver_interface=127.15.11.129
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:16:48.590539 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Bootstrap starting.
I20260809 15:16:48.612953 31392 log.cc:824] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:48.625896 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:16:48.626439 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Bootstrap complete.
I20260809 15:16:48.636018 31392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:48.636929 31392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4956f3a6b4464feb9909b55a840c213f, State: Initialized, Role: FOLLOWER
I20260809 15:16:48.637646 31392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:48.637904 31392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:16:48.638023 31392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:16:48.638258 31392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:16:48.642252 31392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:48.642808 31392 leader_election.cc:302] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4956f3a6b4464feb9909b55a840c213f; no voters:
I20260809 15:16:48.643487 31392 leader_election.cc:288] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:16:48.643620 31396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:16:48.644976 31396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 2 LEADER]: Becoming Leader. State: Replica: 4956f3a6b4464feb9909b55a840c213f, State: Running, Role: LEADER
I20260809 15:16:48.645733 31396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:48.646034 31392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:16:48.648389 31400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4956f3a6b4464feb9909b55a840c213f. Latest consensus state: current_term: 2 leader_uuid: "4956f3a6b4464feb9909b55a840c213f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } } }
I20260809 15:16:48.648793 31400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:48.649319 31397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4956f3a6b4464feb9909b55a840c213f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } } }
I20260809 15:16:48.649603 31397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:48.657829 31405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:16:48.665958 31405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:16:48.667419 31405 catalog_manager.cc:1259] Loaded cluster ID: 09c6a08142fc4dfa84b1c9b17d36d7d5
I20260809 15:16:48.667580 31405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:16:48.670969 31405 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 15:16:48.689963 31405 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 15:16:48.707324 31405 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:16:48.709509 31405 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Loaded TSK: 0
I20260809 15:16:48.711203 31405 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:16:48.962949 31394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:48.963315 31394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:48.963498 31394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:48.971102 31394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:48.971218 31394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:48.971405 31394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:16:48.981431 31394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:35641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=35785
--tserver_master_addrs=127.15.11.190:37409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:48.982730 31394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:48.984113 31394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:48.999017 31424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:48.998965 31425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:48.999509 31427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:48.999858 31394 server_base.cc:1034] running on GCE node
I20260809 15:16:49.000546 31394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:49.001551 31394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:49.002780 31394 hybrid_clock.cc:648] HybridClock initialized: now 1786288609002723 us; error 37 us; skew 500 ppm
I20260809 15:16:49.003101 31394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:49 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:49.010825 31394 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:16:49.012679 31394 webserver.cc:460] Webserver started at http://127.15.11.129:35785/ using document root <none> and password file <none>
I20260809 15:16:49.013439 31394 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:49.013622 31394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:49.019331 31394 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 15:16:49.022151 31437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:49.023499 31394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:16:49.023665 31394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "8eae1d6716974792a2501bb3f006762a"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "6ad38e81dbd6729872a6200a28cd5c2b"
server_key_iv: "4dc43bef3d25b13a93bbfe984c3bcee8"
server_key_version: "encryptionkey@0"
I20260809 15:16:49.024818 31394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:49.038421 31394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:49.042140 31394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:49.042415 31394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:49.043322 31394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:49.044644 31394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:49.044752 31394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:49.044878 31394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:49.044979 31394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:49.078460 31394 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:35641
I20260809 15:16:49.078554 31558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:35641 every 8 connection(s)
I20260809 15:16:49.079990 31394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:16:49.087709 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 31394
I20260809 15:16:49.088495 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:38913
--local_ip_for_outbound_sockets=127.15.11.130
--tserver_master_addrs=127.15.11.190:37409
--webserver_port=45549
--webserver_interface=127.15.11.130
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:16:49 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:49.105099 31563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:49.083203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:52471)
0809 15:16:49.084040 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:49.084090 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:49.085983 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:49.087291 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:49.087303 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:49.088079 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:49.088856 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:49.088883 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:49.090399 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:49.090407 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:49.092435 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:49.092460 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:49.093480 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:49.093489 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:49.093807 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:49.094510 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:49.094540 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:49.097046 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:49.100801 (+ 3755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:49.100809 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:49.100823 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:49.101236 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:49.101697 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:49.101702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:49.101705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:49.101879 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:49.102567 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:49.102575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:49.102884 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 15:16:49.103154 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 15:16:49.103719 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":324,"threads_started":1}
I20260809 15:16:49.105106 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:49.082944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:52471 (local address 127.15.11.190:37409)
0809 15:16:49.083559 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:49.083570 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:49.084224 (+ 654us) server_negotiation.cc:407] Connection header received
0809 15:16:49.086449 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:49.086461 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:49.086756 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:49.087163 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:49.089090 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:49.090237 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:49.092574 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:49.093201 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:49.093337 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:49.097299 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:49.097306 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:49.097318 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:49.097351 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:49.100660 (+ 3309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:49.101402 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:49.101408 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:49.101414 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:49.101570 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:49.101996 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:49.102007 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:49.102009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:49.102431 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:49.102648 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:49.103373 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 15:16:49.103746 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":288,"threads_started":1}
I20260809 15:16:49.108180 31559 heartbeater.cc:347] Connected to a master server at 127.15.11.190:37409
I20260809 15:16:49.108497 31559 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:49.109247 31559 heartbeater.cc:511] Master 127.15.11.190:37409 requested a full tablet report, sending...
I20260809 15:16:49.111642 31351 ts_manager.cc:194] Registered new tserver with Master: 8eae1d6716974792a2501bb3f006762a (127.15.11.129:35641)
I20260809 15:16:49.114027 31351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:52471
W20260809 15:16:49.429335 31564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:49.429656 31564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:49.429754 31564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:49.437004 31564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:49.437100 31564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:49.437219 31564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:16:49.446766 31564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:38913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=45549
--tserver_master_addrs=127.15.11.190:37409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:49.448007 31564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:49.449371 31564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:49.462697 31569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:49.462661 31570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:49.462747 31572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:49.463604 31564 server_base.cc:1034] running on GCE node
I20260809 15:16:49.464214 31564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:49.465137 31564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:49.466369 31564 hybrid_clock.cc:648] HybridClock initialized: now 1786288609466317 us; error 52 us; skew 500 ppm
I20260809 15:16:49.466682 31564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:49 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:49.469656 31564 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:16:49.471057 31564 webserver.cc:460] Webserver started at http://127.15.11.130:45549/ using document root <none> and password file <none>
I20260809 15:16:49.471745 31564 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:49.471904 31564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:49.477022 31564 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:16:49.479456 31579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:49.480800 31564 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:16:49.480983 31564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "bbf475bd1f5e593f144f32fd52df183a"
server_key_iv: "2df77812a8f82acdc0739b5cb0e45190"
server_key_version: "encryptionkey@0"
I20260809 15:16:49.481952 31564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:49.501596 31564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:49.505503 31564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:49.505777 31564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:49.506750 31564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:49.508066 31564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:49.508216 31564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:49.508370 31564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:49.508461 31564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:49.541994 31564 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:38913
I20260809 15:16:49.542083 31704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:38913 every 8 connection(s)
I20260809 15:16:49.543458 31564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:16:49.548053 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 31564
I20260809 15:16:49.548709 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:45043
--local_ip_for_outbound_sockets=127.15.11.131
--tserver_master_addrs=127.15.11.190:37409
--webserver_port=39791
--webserver_interface=127.15.11.131
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:16:49 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:49.563380 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:49.545989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:39879 (local address 127.15.11.190:37409)
0809 15:16:49.546169 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:49.546173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:49.547313 (+ 1140us) server_negotiation.cc:407] Connection header received
0809 15:16:49.549330 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:49.549335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:49.549429 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:49.549540 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:49.550999 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:49.551724 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:49.553295 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:49.553722 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:49.553852 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:49.557521 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:49.557526 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:49.557529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:49.557561 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:49.560458 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:49.561163 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:49.561168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:49.561172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:49.561299 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:49.561744 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:49.561748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:49.561750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:49.562012 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:49.562164 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:49.562804 (+ 640us) server_negotiation.cc:299] Negotiation successful
0809 15:16:49.563029 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:16:49.564371 31707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:49.546401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:39879)
0809 15:16:49.547169 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:49.547206 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:49.549068 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:49.549781 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:49.549790 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:49.550279 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:49.550832 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:49.550849 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:49.551892 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:49.551897 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:49.553145 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:49.553157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:49.553999 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:49.554008 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:49.554239 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:49.554886 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:49.554914 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:49.557299 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:49.560607 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:49.560615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:49.560629 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:49.561036 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:49.561428 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:49.561433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:49.561437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:49.561618 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:49.562138 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:49.562146 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:49.562454 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 15:16:49.562704 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 15:16:49.563199 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":267,"threads_started":1}
I20260809 15:16:49.565840 31705 heartbeater.cc:347] Connected to a master server at 127.15.11.190:37409
I20260809 15:16:49.566152 31705 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:49.567027 31705 heartbeater.cc:511] Master 127.15.11.190:37409 requested a full tablet report, sending...
I20260809 15:16:49.568652 31351 ts_manager.cc:194] Registered new tserver with Master: e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913)
I20260809 15:16:49.569902 31351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:39879
W20260809 15:16:49.886575 31708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:49.886893 31708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:49.887002 31708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:49.894323 31708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:49.894446 31708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:49.894629 31708 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:16:49.903836 31708 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:45043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=39791
--tserver_master_addrs=127.15.11.190:37409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:49.905042 31708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:49.906349 31708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:49.918972 31714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:49.919001 31715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:49.919792 31717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:49.920233 31708 server_base.cc:1034] running on GCE node
I20260809 15:16:49.920809 31708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:49.921663 31708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:49.922861 31708 hybrid_clock.cc:648] HybridClock initialized: now 1786288609922825 us; error 28 us; skew 500 ppm
I20260809 15:16:49.923118 31708 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:49 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:49.926041 31708 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:16:49.927362 31708 webserver.cc:460] Webserver started at http://127.15.11.131:39791/ using document root <none> and password file <none>
I20260809 15:16:49.928153 31708 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:49.928308 31708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:49.933458 31708 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 15:16:49.936157 31725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:49.937537 31708 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 15:16:49.937711 31708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "917ff1ff919f4e879e22e1cb43ca84c4"
format_stamp: "Formatted at 2026-08-09 15:16:47 on dist-test-slave-8w9v"
server_key: "efebdd8fc2ffe180fb4dca0567af819b"
server_key_iv: "bfee33e38836a5562f8d9ff11b03195c"
server_key_version: "encryptionkey@0"
I20260809 15:16:49.938678 31708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:49.955255 31708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:49.959455 31708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:49.959713 31708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:49.960649 31708 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:49.961820 31708 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:49.961941 31708 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:49.962093 31708 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:49.962188 31708 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:49.995281 31708 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:45043
I20260809 15:16:49.995361 31848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:45043 every 8 connection(s)
I20260809 15:16:49.996770 31708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:16:49.998762 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 31708
Aug 09 15:16:50 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288609, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:16:50.014086 31851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:49.999732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35064 (local address 127.15.11.190:37409)
0809 15:16:50.000270 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.000275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.000662 (+ 387us) server_negotiation.cc:407] Connection header received
0809 15:16:50.000892 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.000897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.000969 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.001090 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:50.002662 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.003437 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.004886 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.005391 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.005540 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.008231 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:50.008236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:50.008240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:50.008272 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:50.011239 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.011933 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.011940 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.011943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.012081 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.012521 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.012527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.012535 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.012832 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:50.013110 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.013425 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.013692 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":438,"thread_start_us":292,"threads_started":1}
I20260809 15:16:50.017390 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:49.999536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:48597 (local address 127.15.11.190:37409)
0809 15:16:49.999740 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:49.999746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.001206 (+ 1460us) server_negotiation.cc:407] Connection header received
0809 15:16:50.003077 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.003082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.003159 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.003286 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:50.004865 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.005785 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.007316 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.007773 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.007901 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.011468 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:50.011473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:50.011476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:50.011505 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:50.014192 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.014989 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.014994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.014997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.015129 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.015570 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.015575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.015578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.015827 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:50.015967 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.016861 (+ 894us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.017064 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:16:50.018371 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.000184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:48597)
0809 15:16:50.001059 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.001105 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:50.002813 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:50.003532 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:50.003539 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:50.004026 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:50.004682 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.004700 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.005958 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.005962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.007169 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.007181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.008009 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.008017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.008326 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.008988 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:50.009009 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:50.011230 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:50.014362 (+ 3132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:50.014370 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:50.014385 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:50.014859 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:50.015251 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:50.015256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:50.015260 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:50.015456 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:50.015950 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:50.015957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:50.016418 (+ 461us) client_negotiation.cc:769] Sending connection context
0809 15:16:50.016735 (+ 317us) client_negotiation.cc:240] Negotiation successful
0809 15:16:50.017267 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":357,"threads_started":1}
I20260809 15:16:50.019658 31849 heartbeater.cc:347] Connected to a master server at 127.15.11.190:37409
I20260809 15:16:50.019948 31849 heartbeater.cc:464] Registering TS with master...
I20260809 15:16:50.020747 31849 heartbeater.cc:511] Master 127.15.11.190:37409 requested a full tablet report, sending...
I20260809 15:16:50.022111 31351 ts_manager.cc:194] Registered new tserver with Master: 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
I20260809 15:16:50.022847 31351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:48597
I20260809 15:16:50.028716 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:16:50.035024 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.029552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35066 (local address 127.15.11.190:37409)
0809 15:16:50.029706 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.029710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.030157 (+ 447us) server_negotiation.cc:407] Connection header received
0809 15:16:50.030316 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.030319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.030466 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.030583 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:16:50.030944 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.031676 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.033183 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.033571 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.033658 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.034137 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:16:50.034602 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:16:50.034702 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.034791 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.034850 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:16:50.035827 31351 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:16:50.042835 31351 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:16:50.071048 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.054513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:38913 (local address 127.0.0.1:34988)
0809 15:16:50.055748 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.055794 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:50.055950 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:50.058075 (+ 2125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:50.058086 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:50.058125 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:50.058601 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.058617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.061874 (+ 3257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.061882 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.066239 (+ 4357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.066260 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.069933 (+ 3673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.069942 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.070225 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.070293 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 15:16:50.070456 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 15:16:50.070661 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1008,"thread_start_us":844,"threads_started":1}
I20260809 15:16:50.071105 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.054874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34988 (local address 127.15.11.130:38913)
0809 15:16:50.056350 (+ 1476us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.056358 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.056390 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:16:50.056732 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.056740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.057010 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.057251 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:50.058883 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.061668 (+ 2785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.066472 (+ 4804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.069318 (+ 2846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.069481 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.069731 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.070640 (+ 909us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.070806 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1318,"thread_start_us":1176,"threads_started":1}
I20260809 15:16:50.075112 31632 tablet_service.cc:1467] Processing CreateTablet for tablet 20a3e33cfcc74f7e830304b1607b433f (DEFAULT_TABLE table=test-table [id=252a0a73b9214da3823fa212b04ac3d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:16:50.076642 31632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a3e33cfcc74f7e830304b1607b433f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:50.078425 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.054854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.0.0.1:57398)
0809 15:16:50.066056 (+ 11202us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.066086 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:50.066234 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:50.070928 (+ 4694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:50.070934 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:50.070977 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:50.071506 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.071523 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.074127 (+ 2604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.074133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.076105 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.076116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.077584 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.077590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.077782 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.077812 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:16:50.077925 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 15:16:50.078049 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11033,"thread_start_us":14104,"threads_started":1}
I20260809 15:16:50.079879 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.056602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49536 (local address 127.15.11.131:45043)
0809 15:16:50.070672 (+ 14070us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.070682 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.070718 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:16:50.070838 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.070846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.071073 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.071844 (+ 771us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:50.072106 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.073351 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.075837 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.077417 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.078536 (+ 1119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.078805 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.078883 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.079253 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13897,"thread_start_us":13719,"threads_started":1}
I20260809 15:16:50.080199 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.055766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:45043 (local address 127.0.0.1:49536)
0809 15:16:50.069451 (+ 13685us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.069477 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:50.069631 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:50.071323 (+ 1692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:50.071330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:50.071357 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:50.071792 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.071805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.073518 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.073523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.075662 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.075678 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.077726 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.077734 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.078060 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.078103 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 15:16:50.078207 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:16:50.078370 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13498,"thread_start_us":9964,"threads_started":1}
I20260809 15:16:50.080989 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.056286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.15.11.129:35641)
0809 15:16:50.070256 (+ 13970us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.070267 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.070302 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:16:50.070412 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.070421 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.070683 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.071753 (+ 1070us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:50.072027 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.073977 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.076318 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.077348 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.077510 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.077702 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.080305 (+ 2603us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.080482 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13778,"thread_start_us":13658,"threads_started":1}
I20260809 15:16:50.084256 31777 tablet_service.cc:1467] Processing CreateTablet for tablet 20a3e33cfcc74f7e830304b1607b433f (DEFAULT_TABLE table=test-table [id=252a0a73b9214da3823fa212b04ac3d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:16:50.084853 31488 tablet_service.cc:1467] Processing CreateTablet for tablet 20a3e33cfcc74f7e830304b1607b433f (DEFAULT_TABLE table=test-table [id=252a0a73b9214da3823fa212b04ac3d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:16:50.085805 31777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a3e33cfcc74f7e830304b1607b433f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:50.086721 31488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20a3e33cfcc74f7e830304b1607b433f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:16:50.094029 31866 tablet_bootstrap.cc:492] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Bootstrap starting.
I20260809 15:16:50.097651 31866 tablet_bootstrap.cc:654] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:50.098922 31866 log.cc:824] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:50.098999 31867 tablet_bootstrap.cc:492] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: Bootstrap starting.
I20260809 15:16:50.102313 31869 tablet_bootstrap.cc:492] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Bootstrap starting.
I20260809 15:16:50.103096 31867 tablet_bootstrap.cc:654] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:50.104483 31867 log.cc:824] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:50.105628 31869 tablet_bootstrap.cc:654] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Neither blocks nor log segments found. Creating new log.
I20260809 15:16:50.105806 31866 tablet_bootstrap.cc:492] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: No bootstrap required, opened a new log
I20260809 15:16:50.106082 31866 ts_tablet_manager.cc:1397] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260809 15:16:50.107051 31869 log.cc:824] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:50.107807 31867 tablet_bootstrap.cc:492] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: No bootstrap required, opened a new log
I20260809 15:16:50.108085 31867 ts_tablet_manager.cc:1397] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 15:16:50.110147 31869 tablet_bootstrap.cc:492] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: No bootstrap required, opened a new log
I20260809 15:16:50.110414 31869 ts_tablet_manager.cc:1397] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 15:16:50.116041 31867 raft_consensus.cc:348] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:50.116496 31867 raft_consensus.cc:374] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:50.116652 31867 raft_consensus.cc:729] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 917ff1ff919f4e879e22e1cb43ca84c4, State: Initialized, Role: FOLLOWER
I20260809 15:16:50.116492 31866 raft_consensus.cc:348] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:50.116906 31866 raft_consensus.cc:374] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:50.117005 31866 raft_consensus.cc:729] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c9ee8ebc6e4970a1bd6907bdef3b9d, State: Initialized, Role: FOLLOWER
I20260809 15:16:50.117234 31559 heartbeater.cc:503] Master 127.15.11.190:37409 was elected leader, sending a full tablet report...
I20260809 15:16:50.117416 31867 consensus_queue.cc:260] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:50.117589 31866 consensus_queue.cc:260] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:50.119310 31866 ts_tablet_manager.cc:1428] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260809 15:16:50.119722 31705 heartbeater.cc:503] Master 127.15.11.190:37409 was elected leader, sending a full tablet report...
I20260809 15:16:50.120980 31875 raft_consensus.cc:482] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:16:50.121361 31875 raft_consensus.cc:504] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:50.125404 31869 raft_consensus.cc:348] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:50.125869 31869 raft_consensus.cc:374] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:16:50.125984 31869 raft_consensus.cc:729] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eae1d6716974792a2501bb3f006762a, State: Initialized, Role: FOLLOWER
I20260809 15:16:50.126662 31869 consensus_queue.cc:260] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:50.129334 31849 heartbeater.cc:503] Master 127.15.11.190:37409 was elected leader, sending a full tablet report...
I20260809 15:16:50.130329 31867 ts_tablet_manager.cc:1428] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: Time spent starting tablet: real 0.022s user 0.006s sys 0.015s
I20260809 15:16:50.131002 31875 leader_election.cc:288] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eae1d6716974792a2501bb3f006762a (127.15.11.129:35641), 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
I20260809 15:16:50.134793 31869 ts_tablet_manager.cc:1428] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Time spent starting tablet: real 0.024s user 0.013s sys 0.007s
I20260809 15:16:50.139581 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.125447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:38891 (local address 127.15.11.129:35641)
0809 15:16:50.125721 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.125732 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.132059 (+ 6327us) server_negotiation.cc:407] Connection header received
0809 15:16:50.132381 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.132388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.132467 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.132539 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:50.133538 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.134727 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.137629 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.138616 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.138771 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.138856 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.139195 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.139316 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 15:16:50.139634 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.124920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.15.11.130:38891)
0809 15:16:50.131941 (+ 7021us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.131959 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:50.132161 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:50.132814 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:50.132819 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:50.132850 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:50.133312 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.133329 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.134926 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.134931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.137454 (+ 2523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.137468 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.138903 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.138910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.139025 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.139048 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:16:50.139155 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 15:16:50.139284 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6871,"thread_start_us":6590,"threads_started":1}
I20260809 15:16:50.140679 31509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a3e33cfcc74f7e830304b1607b433f" candidate_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eae1d6716974792a2501bb3f006762a" is_pre_election: true
I20260809 15:16:50.141150 31509 raft_consensus.cc:2454] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7c9ee8ebc6e4970a1bd6907bdef3b9d in term 0.
I20260809 15:16:50.141732 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.132383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:33259 (local address 127.15.11.131:45043)
0809 15:16:50.132622 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.132631 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.133910 (+ 1279us) server_negotiation.cc:407] Connection header received
0809 15:16:50.134187 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.134193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.134268 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.134333 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:16:50.135942 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.137143 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.139608 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.140761 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.140952 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.141035 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.141349 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.141476 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:16:50.141847 31879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.131861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:45043 (local address 127.15.11.130:33259)
0809 15:16:50.133795 (+ 1934us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.133824 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:50.134024 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:50.134538 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:50.134544 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:50.134569 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:16:50.135710 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.135725 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.137688 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.137693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.139443 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:50.139456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.140993 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.141000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:50.141174 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.141199 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:16:50.141322 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 15:16:50.141445 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1778,"thread_start_us":4487,"threads_started":1}
I20260809 15:16:50.141867 31581 leader_election.cc:302] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eae1d6716974792a2501bb3f006762a, e7c9ee8ebc6e4970a1bd6907bdef3b9d; no voters:
I20260809 15:16:50.142328 31875 raft_consensus.cc:2781] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:16:50.142503 31875 raft_consensus.cc:482] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:16:50.142625 31875 raft_consensus.cc:3037] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:50.142867 31798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a3e33cfcc74f7e830304b1607b433f" candidate_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" is_pre_election: true
I20260809 15:16:50.143332 31798 raft_consensus.cc:2454] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7c9ee8ebc6e4970a1bd6907bdef3b9d in term 0.
I20260809 15:16:50.145368 31875 raft_consensus.cc:504] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:50.146039 31875 leader_election.cc:288] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [CANDIDATE]: Term 1 election: Requested vote from peers 8eae1d6716974792a2501bb3f006762a (127.15.11.129:35641), 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
I20260809 15:16:50.146440 31509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a3e33cfcc74f7e830304b1607b433f" candidate_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eae1d6716974792a2501bb3f006762a"
I20260809 15:16:50.146561 31798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20a3e33cfcc74f7e830304b1607b433f" candidate_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "917ff1ff919f4e879e22e1cb43ca84c4"
I20260809 15:16:50.146647 31509 raft_consensus.cc:3037] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:50.146734 31798 raft_consensus.cc:3037] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:16:50.149930 31509 raft_consensus.cc:2454] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7c9ee8ebc6e4970a1bd6907bdef3b9d in term 1.
I20260809 15:16:50.149935 31798 raft_consensus.cc:2454] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7c9ee8ebc6e4970a1bd6907bdef3b9d in term 1.
I20260809 15:16:50.150331 31581 leader_election.cc:302] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eae1d6716974792a2501bb3f006762a, e7c9ee8ebc6e4970a1bd6907bdef3b9d; no voters:
I20260809 15:16:50.150658 31875 raft_consensus.cc:2781] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:16:50.151100 31875 raft_consensus.cc:686] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 1 LEADER]: Becoming Leader. State: Replica: e7c9ee8ebc6e4970a1bd6907bdef3b9d, State: Running, Role: LEADER
I20260809 15:16:50.151546 31875 consensus_queue.cc:237] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:50.156281 31350 catalog_manager.cc:5495] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d reported cstate change: term changed from 0 to 1, leader changed from <none> to e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130). New cstate: current_term: 1 leader_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } health_report { overall_health: UNKNOWN } } }
I20260809 15:16:50.192775 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.187306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34994 (local address 127.15.11.130:38913)
0809 15:16:50.187572 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.187576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.187673 (+ 97us) server_negotiation.cc:407] Connection header received
0809 15:16:50.187851 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.187855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.187936 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.188055 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:16:50.188471 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.189201 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.190638 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.191034 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.191123 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.191597 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:16:50.192076 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:16:50.192280 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.192335 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.192427 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 15:16:50.200946 31798 raft_consensus.cc:1261] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [term 1 FOLLOWER]: Refusing update from remote peer e7c9ee8ebc6e4970a1bd6907bdef3b9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:16:50.200950 31509 raft_consensus.cc:1261] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Refusing update from remote peer e7c9ee8ebc6e4970a1bd6907bdef3b9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:16:50.201957 31880 consensus_queue.cc:1035] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:16:50.202311 31875 consensus_queue.cc:1035] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:16:50.230638 31350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:16:50.230919 31350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:16:50.234388 31350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Sending DeleteTablet for 3 replicas of tablet 20a3e33cfcc74f7e830304b1607b433f
I20260809 15:16:50.235617 31488 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a3e33cfcc74f7e830304b1607b433f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:16:50 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:57398
I20260809 15:16:50.235625 31632 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a3e33cfcc74f7e830304b1607b433f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:16:50 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:34988
I20260809 15:16:50.235782 31777 tablet_service.cc:1514] Processing DeleteTablet for tablet 20a3e33cfcc74f7e830304b1607b433f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:16:50 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:49536
I20260809 15:16:50.242622 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 31394
I20260809 15:16:50.248145 31894 tablet_replica.cc:330] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: stopping tablet replica
I20260809 15:16:50.248695 31894 raft_consensus.cc:2229] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:16:50.249558 31894 raft_consensus.cc:2258] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:16:50.250356 31895 tablet_replica.cc:330] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: stopping tablet replica
I20260809 15:16:50.250962 31897 tablet_replica.cc:330] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: stopping tablet replica
I20260809 15:16:50.251397 31897 raft_consensus.cc:2229] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:16:50.251916 31897 raft_consensus.cc:2258] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:16:50.252709 31894 ts_tablet_manager.cc:1905] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:16:50.253401 31895 raft_consensus.cc:2229] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:16:50.253883 31895 raft_consensus.cc:2258] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:16:50.255193 31897 ts_tablet_manager.cc:1905] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:16:50.259271 31895 ts_tablet_manager.cc:1905] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:16:50.261461 31894 ts_tablet_manager.cc:1918] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:16:50.261701 31894 log.cc:1196] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/20a3e33cfcc74f7e830304b1607b433f
I20260809 15:16:50.262403 31894 ts_tablet_manager.cc:1939] T 20a3e33cfcc74f7e830304b1607b433f P e7c9ee8ebc6e4970a1bd6907bdef3b9d: Deleting consensus metadata
I20260809 15:16:50.264076 31864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.243889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57402 (local address 127.15.11.129:35641)
0809 15:16:50.244300 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.244309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.244720 (+ 411us) server_negotiation.cc:407] Connection header received
0809 15:16:50.244796 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.244805 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.244900 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.245021 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:50.247514 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.248669 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.250803 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.251476 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.251605 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.255765 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:50.255779 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:50.255795 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:50.255836 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:50.260182 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.261085 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.261094 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.261099 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.261281 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.261778 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.261785 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.261788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.262329 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:50.262633 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.263509 (+ 876us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.263730 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260}
I20260809 15:16:50.264823 31336 catalog_manager.cc:4841] TS e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913): tablet 20a3e33cfcc74f7e830304b1607b433f (table test-table [id=252a0a73b9214da3823fa212b04ac3d4]) successfully deleted
I20260809 15:16:50.265363 31552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:50.266811 31897 ts_tablet_manager.cc:1918] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:16:50.267009 31897 log.cc:1196] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/20a3e33cfcc74f7e830304b1607b433f
I20260809 15:16:50.267675 31897 ts_tablet_manager.cc:1939] T 20a3e33cfcc74f7e830304b1607b433f P 917ff1ff919f4e879e22e1cb43ca84c4: Deleting consensus metadata
I20260809 15:16:50.269781 31332 catalog_manager.cc:4841] TS 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043): tablet 20a3e33cfcc74f7e830304b1607b433f (table test-table [id=252a0a73b9214da3823fa212b04ac3d4]) successfully deleted
I20260809 15:16:50.381429 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 31394
W20260809 15:16:50.400913 31333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.11.129:35641 (error 108)
I20260809 15:16:50.401136 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 31564
I20260809 15:16:50.403026 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.402502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.0.0.1:57414)
0809 15:16:50.402713 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.402832 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 15:16:50.403327 31333 catalog_manager.cc:4584] TS 8eae1d6716974792a2501bb3f006762a (127.15.11.129:35641): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 20a3e33cfcc74f7e830304b1607b433f: Network error: Client connection negotiation failed: client connection to 127.15.11.129:35641: connect: Connection refused (error 111)
I20260809 15:16:50.417735 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.402124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35010 (local address 127.15.11.130:38913)
0809 15:16:50.402331 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.402338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.402518 (+ 180us) server_negotiation.cc:407] Connection header received
0809 15:16:50.402743 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.402750 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.402834 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.402954 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:50.404839 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.406056 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.407465 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.408175 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.408299 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.410748 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:50.410763 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:50.410800 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:50.410840 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:50.414857 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.415521 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.415530 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.415534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.415715 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.416048 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.416053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.416057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.416537 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:50.416846 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.417120 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.417334 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:16:50.418587 31699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:50.457036 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.456549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.0.0.1:57430)
0809 15:16:50.456773 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.456880 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 15:16:50.458122 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.457758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.0.0.1:57442)
0809 15:16:50.457925 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.457993 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 15:16:50.518510 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.518066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.0.0.1:57454)
0809 15:16:50.518261 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.518344 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 15:16:50.519524 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.519195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.0.0.1:57470)
0809 15:16:50.519336 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.519414 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 15:16:50.530571 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 31564
I20260809 15:16:50.553148 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 31708
I20260809 15:16:50.567123 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.553902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49538 (local address 127.15.11.131:45043)
0809 15:16:50.554087 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:50.554094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:50.554157 (+ 63us) server_negotiation.cc:407] Connection header received
0809 15:16:50.554446 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:50.554450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:50.554570 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:50.554672 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:50.556516 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.557243 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.558602 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:50.559002 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:50.559093 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:50.561750 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:50.561764 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:50.561779 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:50.561816 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:50.564699 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.565322 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.565342 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.565344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.565492 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:50.565808 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:50.565811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:50.565813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:50.566134 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:50.566359 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:50.566617 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 15:16:50.566771 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:16:50.567766 31842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:50.599875 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.599441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.0.0.1:57472)
0809 15:16:50.599636 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.599727 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 15:16:50.600983 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:50.600596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:35641 (local address 127.0.0.1:57478)
0809 15:16:50.600796 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:50.600869 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:35641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 15:16:50.668063 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 31708
I20260809 15:16:50.690428 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 31311
I20260809 15:16:50.691648 31387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:16:50.804680 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 31311
I20260809 15:16:50.828924 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37409
--webserver_interface=127.15.11.190
--webserver_port=36943
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:37409
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:51.153985 31903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:51.154242 31903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:51.154335 31903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:51.161247 31903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:16:51.161370 31903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:51.161456 31903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:51.161518 31903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:16:51.161582 31903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:16:51.170358 31903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:37409
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:37409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=36943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:51.171568 31903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:51.172838 31903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:51.184021 31908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:51.184063 31909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:51.184109 31911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:51.188220 31903 server_base.cc:1034] running on GCE node
I20260809 15:16:51.188952 31903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:51.190244 31903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:51.191447 31903 hybrid_clock.cc:648] HybridClock initialized: now 1786288611191378 us; error 54 us; skew 500 ppm
I20260809 15:16:51.191789 31903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:51 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288611, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:51.194857 31903 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:16:51.196296 31903 webserver.cc:460] Webserver started at http://127.15.11.190:36943/ using document root <none> and password file <none>
I20260809 15:16:51.196973 31903 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:51.197115 31903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:51.202387 31903 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 15:16:51.205377 31918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:51.206657 31903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 15:16:51.206842 31903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "4956f3a6b4464feb9909b55a840c213f"
format_stamp: "Formatted at 2026-08-09 15:16:45 on dist-test-slave-8w9v"
server_key: "d89fda388099937b8c83ce38b675672e"
server_key_iv: "9d1d69c8d4ca4b7be2853b3cc8f4d08f"
server_key_version: "encryptionkey@0"
I20260809 15:16:51.207811 31903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:51.220453 31903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:51.224355 31903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:51.224609 31903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:51.243359 31903 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:37409
I20260809 15:16:51.243379 31973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:37409 every 8 connection(s)
I20260809 15:16:51.244870 31903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:16:51.254347 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 31903
I20260809 15:16:51.255388 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:35641
--local_ip_for_outbound_sockets=127.15.11.129
--tserver_master_addrs=127.15.11.190:37409
--webserver_port=35785
--webserver_interface=127.15.11.129
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:16:51.256976 31974 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:16:51.260911 31974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Bootstrap starting.
I20260809 15:16:51.290154 31974 log.cc:824] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:51.311715 31974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:16:51.312404 31974 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f: Bootstrap complete.
I20260809 15:16:51.326835 31974 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:51.328042 31974 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4956f3a6b4464feb9909b55a840c213f, State: Initialized, Role: FOLLOWER
I20260809 15:16:51.328791 31974 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:51.329053 31974 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:16:51.329171 31974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:16:51.329353 31974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:16:51.335937 31974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:51.336603 31974 leader_election.cc:302] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4956f3a6b4464feb9909b55a840c213f; no voters:
I20260809 15:16:51.337414 31974 leader_election.cc:288] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 15:16:51.337682 31979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:16:51.344827 31979 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [term 3 LEADER]: Becoming Leader. State: Replica: 4956f3a6b4464feb9909b55a840c213f, State: Running, Role: LEADER
I20260809 15:16:51.345350 31979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } }
I20260809 15:16:51.345844 31974 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:16:51.348441 31981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4956f3a6b4464feb9909b55a840c213f. Latest consensus state: current_term: 3 leader_uuid: "4956f3a6b4464feb9909b55a840c213f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } } }
I20260809 15:16:51.348862 31981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:51.349406 31980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4956f3a6b4464feb9909b55a840c213f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4956f3a6b4464feb9909b55a840c213f" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 37409 } } }
I20260809 15:16:51.349825 31980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4956f3a6b4464feb9909b55a840c213f [sys.catalog]: This master's current role is: LEADER
I20260809 15:16:51.356747 31985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:16:51.375680 31985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:16:51.381850 31985 catalog_manager.cc:1259] Loaded cluster ID: 09c6a08142fc4dfa84b1c9b17d36d7d5
I20260809 15:16:51.386235 31985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 15:16:51.390065 31985 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7feac41ad9ad google::LogMessage::Fail() at ??:0
@ 0x7feac41b088a google::LogMessage::SendToLog() at ??:0
@ 0x7feac41ad4de google::LogMessage::Flush() at ??:0
@ 0x7feac41aea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fead9be0512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fead9bdd976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fead9c65610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fead9c65439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7feac6df9345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7feac6dfc91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7feac6dfc799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7feac6dd7eaf kudu::Thread::SuperviseThread() at ??:0
I20260809 15:16:51.484651 32008 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7feaca0b16db start_thread at ??:0
@ 0x7feac304761f clone at ??:0
W20260809 15:16:51.735345 31976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:51.735605 31976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:51.735743 31976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:51.742810 31976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:51.742905 31976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:51.743090 31976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:16:51.755895 31976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:35641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=35785
--tserver_master_addrs=127.15.11.190:37409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:51.757272 31976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:51.758663 31976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:51.771759 32021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:51.772367 32020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:51.772482 32024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:51.773100 31976 server_base.cc:1034] running on GCE node
I20260809 15:16:51.773706 31976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:51.774677 31976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:51.776021 31976 hybrid_clock.cc:648] HybridClock initialized: now 1786288611775956 us; error 42 us; skew 500 ppm
I20260809 15:16:51.776415 31976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:51 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288611, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:51.779495 31976 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:16:51.781082 31976 webserver.cc:460] Webserver started at http://127.15.11.129:35785/ using document root <none> and password file <none>
I20260809 15:16:51.781736 31976 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:51.781863 31976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:51.788295 31976 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 15:16:51.794013 32031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:51.795382 31976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:16:51.795547 31976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "8eae1d6716974792a2501bb3f006762a"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "6ad38e81dbd6729872a6200a28cd5c2b"
server_key_iv: "4dc43bef3d25b13a93bbfe984c3bcee8"
server_key_version: "encryptionkey@0"
I20260809 15:16:51.796517 31976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:51.816524 31976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:51.820232 31976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:51.820500 31976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:51.821437 31976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:51.823105 32038 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 15:16:51.828306 31976 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 15:16:51.828428 31976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 15:16:51.828576 31976 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 15:16:51.831534 31976 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 15:16:51.831641 31976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260809 15:16:51.831957 32038 tablet_bootstrap.cc:492] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Bootstrap starting.
I20260809 15:16:51.840427 32038 log.cc:824] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:16:51.854777 32038 tablet_bootstrap.cc:492] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:16:51.855734 32038 tablet_bootstrap.cc:492] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Bootstrap complete.
I20260809 15:16:51.856397 32038 ts_tablet_manager.cc:1397] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Time spent bootstrapping tablet: real 0.025s user 0.017s sys 0.006s
I20260809 15:16:51.866613 32038 raft_consensus.cc:348] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:51.867640 32038 raft_consensus.cc:729] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eae1d6716974792a2501bb3f006762a, State: Initialized, Role: FOLLOWER
I20260809 15:16:51.868357 32038 consensus_queue.cc:260] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:51.869927 32038 ts_tablet_manager.cc:1428] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260809 15:16:51.877856 31976 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:35641
I20260809 15:16:51.877936 32156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:35641 every 8 connection(s)
I20260809 15:16:51.879033 31976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:16:51.883512 32159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:51.881727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:42857)
0809 15:16:51.882459 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:51.882671 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":304,"threads_started":1}
W20260809 15:16:51.884100 32157 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:37409 (0 consecutive failures): Network error: Failed to ping master at 127.15.11.190:37409: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
I20260809 15:16:51.884972 32159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:51.884669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:60947)
0809 15:16:51.884802 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:51.884874 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 15:16:51.885797 32159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:51.885457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:33063)
0809 15:16:51.885591 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:51.885659 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:16:51.886020 32157 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:16:51.886400 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 31976
I20260809 15:16:51.887007 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:38913
--local_ip_for_outbound_sockets=127.15.11.130
--tserver_master_addrs=127.15.11.190:37409
--webserver_port=45549
--webserver_interface=127.15.11.130
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:52.225615 32160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:52.225936 32160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:52.226099 32160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:52.233175 32160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:52.233278 32160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:52.233484 32160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:16:52.243202 32160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:38913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=45549
--tserver_master_addrs=127.15.11.190:37409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:52.244530 32160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:52.245857 32160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:52.258940 32166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:52.258994 32170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:52.258940 32168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:52.259927 32160 server_base.cc:1034] running on GCE node
I20260809 15:16:52.260599 32160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:52.261533 32160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:52.262753 32160 hybrid_clock.cc:648] HybridClock initialized: now 1786288612262682 us; error 59 us; skew 500 ppm
I20260809 15:16:52.263075 32160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:52 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:52.266342 32160 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:16:52.267849 32160 webserver.cc:460] Webserver started at http://127.15.11.130:45549/ using document root <none> and password file <none>
I20260809 15:16:52.268581 32160 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:52.268684 32160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:52.273964 32160 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:16:52.276690 32177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:52.278002 32160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:16:52.278183 32160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d"
format_stamp: "Formatted at 2026-08-09 15:16:46 on dist-test-slave-8w9v"
server_key: "bbf475bd1f5e593f144f32fd52df183a"
server_key_iv: "2df77812a8f82acdc0739b5cb0e45190"
server_key_version: "encryptionkey@0"
I20260809 15:16:52.279156 32160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:52.302115 32160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:52.306043 32160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:52.306325 32160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:52.307305 32160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:52.308715 32160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:52.308838 32160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:52.308974 32160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:52.309064 32160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:52.342712 32160 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:38913
I20260809 15:16:52.342798 32297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:38913 every 8 connection(s)
I20260809 15:16:52.344264 32160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:16:52.349016 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:52.347192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:58319)
0809 15:16:52.347926 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:52.348222 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":290,"threads_started":1}
W20260809 15:16:52.349646 32299 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:37409 (0 consecutive failures): Network error: Failed to ping master at 127.15.11.190:37409: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
I20260809 15:16:52.350404 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:52.350105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:33363)
0809 15:16:52.350239 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:52.350301 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 15:16:52.351296 32301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:52.350973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:36501)
0809 15:16:52.351103 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:52.351173 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 15:16:52.351539 32299 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:16:52.354082 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 32160
I20260809 15:16:52.354707 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:45043
--local_ip_for_outbound_sockets=127.15.11.131
--tserver_master_addrs=127.15.11.190:37409
--webserver_port=39791
--webserver_interface=127.15.11.131
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:16:52.689002 32302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:16:52.689308 32302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:16:52.689492 32302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:16:52.696485 32302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:16:52.696602 32302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:16:52.696787 32302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:16:52.706133 32302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:45043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=39791
--tserver_master_addrs=127.15.11.190:37409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:16:52.707351 32302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:16:52.708731 32302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:16:52.722282 32309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:52.722625 32307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:16:52.722664 32311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:16:52.723012 32302 server_base.cc:1034] running on GCE node
I20260809 15:16:52.723603 32302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:16:52.724504 32302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:16:52.725692 32302 hybrid_clock.cc:648] HybridClock initialized: now 1786288612725634 us; error 59 us; skew 500 ppm
I20260809 15:16:52.726027 32302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:16:52 dist-test-slave-8w9v krb5kdc[30694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288612, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:16:52.729113 32302 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:16:52.730604 32302 webserver.cc:460] Webserver started at http://127.15.11.131:39791/ using document root <none> and password file <none>
I20260809 15:16:52.731272 32302 fs_manager.cc:359] Metadata directory not provided
I20260809 15:16:52.731424 32302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:16:52.736716 32302 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:16:52.739409 32318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:52.740665 32302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:16:52.740829 32302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "917ff1ff919f4e879e22e1cb43ca84c4"
format_stamp: "Formatted at 2026-08-09 15:16:47 on dist-test-slave-8w9v"
server_key: "efebdd8fc2ffe180fb4dca0567af819b"
server_key_iv: "bfee33e38836a5562f8d9ff11b03195c"
server_key_version: "encryptionkey@0"
I20260809 15:16:52.741806 32302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:16:52.776161 32302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:16:52.780083 32302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:16:52.780400 32302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:16:52.781399 32302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:16:52.782773 32302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:16:52.782902 32302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:52.783016 32302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:16:52.783100 32302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:16:52.816006 32302 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:45043
I20260809 15:16:52.816073 32439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:45043 every 8 connection(s)
I20260809 15:16:52.817538 32302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:16:52.822234 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:52.820487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:32829)
0809 15:16:52.821221 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:52.821450 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":286,"threads_started":1}
W20260809 15:16:52.822872 32440 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:37409 (0 consecutive failures): Network error: Failed to ping master at 127.15.11.190:37409: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
I20260809 15:16:52.823596 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 32302
I20260809 15:16:52.823657 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:52.823345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:57801)
0809 15:16:52.823503 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:52.823563 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 15:16:52.824572 32442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:52.824242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:59681)
0809 15:16:52.824382 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:52.824456 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 15:16:52.824820 32440 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:16:52.887229 32444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:52.886466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:36381)
0809 15:16:52.886879 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:52.886987 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":188,"threads_started":1}
I20260809 15:16:53.034929 32445 raft_consensus.cc:482] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:16:53.035257 32445 raft_consensus.cc:504] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:53.037034 32445 leader_election.cc:288] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913), 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
Aug 09 15:16:53 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288611, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
Aug 09 15:16:53 dist-test-slave-8w9v krb5kdc[30694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288611, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:16:53.065972 32444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:53.037372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:38913 (local address 127.15.11.129:52583)
0809 15:16:53.037691 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:53.037726 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:53.038364 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:53.042551 (+ 4187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:53.042564 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:53.043569 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:53.044822 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:53.044867 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:53.047184 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:53.047190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:53.049663 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:53.049679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:53.051065 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:53.051072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:53.051430 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:53.052438 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:53.052484 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:53.056357 (+ 3873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:53.061122 (+ 4765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:53.061132 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:53.061160 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:53.062177 (+ 1017us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:53.062705 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:53.062711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:53.062714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:53.063064 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:53.064063 (+ 999us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:53.064071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:53.064581 (+ 510us) client_negotiation.cc:769] Sending connection context
0809 15:16:53.064913 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 15:16:53.065605 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":836,"spinlock_wait_cycles":92160}
I20260809 15:16:53.065972 32450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:53.037385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:45043 (local address 127.15.11.129:44751)
0809 15:16:53.039074 (+ 1689us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:53.039094 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:16:53.039219 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:16:53.042551 (+ 3332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:16:53.042564 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:16:53.043569 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:16:53.044822 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:53.044852 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:53.047188 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:53.047192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:53.049663 (+ 2471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:16:53.049679 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:16:53.051013 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:16:53.051023 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:16:53.051433 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:53.052438 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:16:53.052477 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:16:53.056357 (+ 3880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:16:53.061122 (+ 4765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:53.061132 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:53.061160 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:53.062177 (+ 1017us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:53.062705 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:16:53.062710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:16:53.062714 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:16:53.063064 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:16:53.064063 (+ 999us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:16:53.064071 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:16:53.064581 (+ 510us) client_negotiation.cc:769] Sending connection context
0809 15:16:53.064913 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 15:16:53.065494 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1565,"mutex_wait_us":1034,"spinlock_wait_cycles":35456,"thread_start_us":828,"threads_started":1}
I20260809 15:16:53.066032 32446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:53.037638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:52583 (local address 127.15.11.130:38913)
0809 15:16:53.039185 (+ 1547us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:53.039201 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:53.039236 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:16:53.041482 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:53.041495 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:53.041994 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:53.042371 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:53.045052 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:53.046958 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:53.049808 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:53.050786 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:53.050981 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:53.056681 (+ 5700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:53.056692 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:53.056710 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:53.056760 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:53.060956 (+ 4196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:53.062332 (+ 1376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:53.062342 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:53.062350 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:53.062586 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:53.063210 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:53.063215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:53.063219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:53.063927 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:53.064109 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:53.065185 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0809 15:16:53.065605 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1383,"thread_start_us":1268,"threads_started":1}
I20260809 15:16:53.067521 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:53.037683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:44751 (local address 127.15.11.131:45043)
0809 15:16:53.038515 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 15:16:53.038536 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:16:53.039227 (+ 691us) server_negotiation.cc:407] Connection header received
0809 15:16:53.041484 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:16:53.041495 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:16:53.041994 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:16:53.042371 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:16:53.045052 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:53.046958 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:53.049808 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:16:53.050787 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:16:53.050986 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:16:53.056681 (+ 5695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:16:53.056692 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:16:53.056710 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:16:53.056760 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:16:53.060952 (+ 4192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:53.062332 (+ 1380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:53.062342 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:53.062350 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:53.062580 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:16:53.063210 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:16:53.063215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:16:53.063219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:16:53.063918 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:16:53.064106 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 15:16:53.065185 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0809 15:16:53.065605 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":405,"threads_started":1}
W20260809 15:16:53.069413 32035 leader_election.cc:341] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913): Not found: Tablet not found: 20a3e33cfcc74f7e830304b1607b433f
W20260809 15:16:53.070043 32032 leader_election.cc:341] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043): Not found: Tablet not found: 20a3e33cfcc74f7e830304b1607b433f
I20260809 15:16:53.070209 32032 leader_election.cc:302] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eae1d6716974792a2501bb3f006762a; no voters: 917ff1ff919f4e879e22e1cb43ca84c4, e7c9ee8ebc6e4970a1bd6907bdef3b9d
I20260809 15:16:53.070607 32445 raft_consensus.cc:2726] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:16:53.352906 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:53.352035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:59567)
0809 15:16:53.352538 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:53.352677 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":266,"threads_started":1}
I20260809 15:16:53.826067 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:53.825334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:33913)
0809 15:16:53.825765 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:53.825877 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":219,"threads_started":1}
I20260809 15:16:53.888734 32453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:53.888062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:35565)
0809 15:16:53.888487 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:53.888580 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":212,"threads_started":1}
I20260809 15:16:54.354470 32454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:54.353785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:46923)
0809 15:16:54.354217 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:54.354299 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":224,"threads_started":1}
I20260809 15:16:54.827759 32455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:54.827026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:39887)
0809 15:16:54.827511 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:54.827590 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":285,"threads_started":1}
I20260809 15:16:54.874782 32456 raft_consensus.cc:482] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:16:54.875028 32456 raft_consensus.cc:504] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:54.875739 32456 leader_election.cc:288] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913), 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
W20260809 15:16:54.876750 32035 leader_election.cc:341] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913): Not found: Tablet not found: 20a3e33cfcc74f7e830304b1607b433f
W20260809 15:16:54.876983 32032 leader_election.cc:341] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043): Not found: Tablet not found: 20a3e33cfcc74f7e830304b1607b433f
I20260809 15:16:54.877118 32032 leader_election.cc:302] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eae1d6716974792a2501bb3f006762a; no voters: 917ff1ff919f4e879e22e1cb43ca84c4, e7c9ee8ebc6e4970a1bd6907bdef3b9d
I20260809 15:16:54.877429 32456 raft_consensus.cc:2726] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:16:54.890213 32458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:54.889517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:35929)
0809 15:16:54.889955 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:54.890035 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":224,"threads_started":1}
I20260809 15:16:55.355880 32459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:55.355259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:53073)
0809 15:16:55.355655 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:55.355734 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":205,"threads_started":1}
I20260809 15:16:55.829260 32460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:55.828592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:55611)
0809 15:16:55.829037 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:55.829118 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":250,"threads_started":1}
I20260809 15:16:55.891613 32461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:55.890966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:40005)
0809 15:16:55.891363 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:55.891439 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":219,"threads_started":1}
I20260809 15:16:56.357542 32462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:56.356759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:38641)
0809 15:16:56.357214 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:56.357340 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":237,"threads_started":1}
I20260809 15:16:56.830852 32463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:56.830155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:33663)
0809 15:16:56.830578 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:56.830681 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260809 15:16:56.893385 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:56.892624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:46487)
0809 15:16:56.893082 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:56.893193 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":210,"threads_started":1}
I20260809 15:16:57.359088 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:57.358436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:42431)
0809 15:16:57.358838 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:57.358925 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":213,"threads_started":1}
I20260809 15:16:57.540920 32466 raft_consensus.cc:482] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:16:57.541162 32466 raft_consensus.cc:504] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:16:57.541841 32466 leader_election.cc:288] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913), 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
W20260809 15:16:57.542776 32032 leader_election.cc:341] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043): Not found: Tablet not found: 20a3e33cfcc74f7e830304b1607b433f
W20260809 15:16:57.542955 32035 leader_election.cc:341] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913): Not found: Tablet not found: 20a3e33cfcc74f7e830304b1607b433f
I20260809 15:16:57.543090 32035 leader_election.cc:302] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eae1d6716974792a2501bb3f006762a; no voters: 917ff1ff919f4e879e22e1cb43ca84c4, e7c9ee8ebc6e4970a1bd6907bdef3b9d
I20260809 15:16:57.543359 32466 raft_consensus.cc:2726] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:16:57.832383 32467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:57.831669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:60041)
0809 15:16:57.832093 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:57.832224 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":213,"threads_started":1}
I20260809 15:16:57.895305 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:57.894302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:50815)
0809 15:16:57.894923 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:57.895052 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":306,"threads_started":1}
I20260809 15:16:58.360644 32469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:58.359960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:50947)
0809 15:16:58.360411 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:58.360486 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":253,"threads_started":1}
I20260809 15:16:58.833873 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:58.833191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:32987)
0809 15:16:58.833647 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:58.833717 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":260,"threads_started":1}
I20260809 15:16:58.897063 32471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:58.896332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:55007)
0809 15:16:58.896778 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:58.896859 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":242,"threads_started":1}
I20260809 15:16:59.362213 32472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:59.361450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:56319)
0809 15:16:59.361954 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:59.362040 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":218,"threads_started":1}
I20260809 15:16:59.835575 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:59.834697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:40441)
0809 15:16:59.835220 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:59.835325 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":227,"threads_started":1}
I20260809 15:16:59.898540 32474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:16:59.897864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:57985)
0809 15:16:59.898305 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 15:16:59.898383 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":213,"threads_started":1}
I20260809 15:17:00.363744 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:00.363096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:48245)
0809 15:17:00.363492 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:00.363569 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":197,"threads_started":1}
I20260809 15:17:00.837139 32477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:00.836489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:55231)
0809 15:17:00.836872 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:00.836965 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":199,"threads_started":1}
I20260809 15:17:00.900086 32478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:00.899357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:49231)
0809 15:17:00.899818 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:00.899910 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":246,"threads_started":1}
I20260809 15:17:01.365401 32479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:01.364667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:45433)
0809 15:17:01.365138 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:01.365227 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":224,"threads_started":1}
I20260809 15:17:01.838840 32480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:01.838044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:38493)
0809 15:17:01.838602 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:01.838683 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":301,"threads_started":1}
I20260809 15:17:01.901841 32481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:01.901064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:36043)
0809 15:17:01.901584 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:01.901669 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":307,"threads_started":1}
I20260809 15:17:02.367177 32482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:02.366404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:54763)
0809 15:17:02.366903 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:02.366993 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":252,"threads_started":1}
I20260809 15:17:02.546452 32483 raft_consensus.cc:482] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:17:02.546654 32483 raft_consensus.cc:504] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7c9ee8ebc6e4970a1bd6907bdef3b9d" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 38913 } } peers { permanent_uuid: "8eae1d6716974792a2501bb3f006762a" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 35641 } } peers { permanent_uuid: "917ff1ff919f4e879e22e1cb43ca84c4" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 45043 } }
I20260809 15:17:02.547304 32483 leader_election.cc:288] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913), 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043)
W20260809 15:17:02.548337 32032 leader_election.cc:341] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 917ff1ff919f4e879e22e1cb43ca84c4 (127.15.11.131:45043): Not found: Tablet not found: 20a3e33cfcc74f7e830304b1607b433f
W20260809 15:17:02.548568 32035 leader_election.cc:341] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e7c9ee8ebc6e4970a1bd6907bdef3b9d (127.15.11.130:38913): Not found: Tablet not found: 20a3e33cfcc74f7e830304b1607b433f
I20260809 15:17:02.548712 32035 leader_election.cc:302] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eae1d6716974792a2501bb3f006762a; no voters: 917ff1ff919f4e879e22e1cb43ca84c4, e7c9ee8ebc6e4970a1bd6907bdef3b9d
I20260809 15:17:02.548969 32483 raft_consensus.cc:2726] T 20a3e33cfcc74f7e830304b1607b433f P 8eae1d6716974792a2501bb3f006762a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:17:02.840536 32484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:02.839777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:52961)
0809 15:17:02.840276 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:02.840357 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":246,"threads_started":1}
I20260809 15:17:02.903502 32485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:02.902699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:48345)
0809 15:17:02.903189 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:02.903298 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":250,"threads_started":1}
I20260809 15:17:03.369257 32486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:03.368414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:46907)
0809 15:17:03.368990 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:03.369098 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":279,"threads_started":1}
I20260809 15:17:03.842041 32487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:03.841389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:42229)
0809 15:17:03.841794 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:03.841881 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":212,"threads_started":1}
I20260809 15:17:03.905014 32489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:03.904303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:43749)
0809 15:17:03.904751 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:03.904829 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":221,"threads_started":1}
I20260809 15:17:04.370952 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:04.370235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:46155)
0809 15:17:04.370686 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:04.370769 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":244,"threads_started":1}
I20260809 15:17:04.843595 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:04.842872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:34461)
0809 15:17:04.843346 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:04.843425 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":241,"threads_started":1}
I20260809 15:17:04.906499 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:04.905816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:36667)
0809 15:17:04.906241 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:04.906329 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":243,"threads_started":1}
I20260809 15:17:05.372653 32494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:05.371802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:38223)
0809 15:17:05.372352 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:05.372479 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":226,"threads_started":1}
I20260809 15:17:05.845259 32495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:05.844468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:52859)
0809 15:17:05.844934 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:05.845036 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":243,"threads_started":1}
I20260809 15:17:05.908201 32496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:05.907400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:38061)
0809 15:17:05.907847 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:05.907952 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":246,"threads_started":1}
I20260809 15:17:06.374293 32497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:06.373486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:43123)
0809 15:17:06.373970 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:06.374088 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":230,"threads_started":1}
I20260809 15:17:06.846810 32498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:06.846016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:55075)
0809 15:17:06.846427 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:06.846540 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":218,"threads_started":1}
I20260809 15:17:06.910085 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:06.909323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.129:44365)
0809 15:17:06.909834 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:06.909911 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":284,"threads_started":1}
I20260809 15:17:07.375958 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:07.375317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.130:33573)
0809 15:17:07.375703 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:07.375795 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":193,"threads_started":1}
I20260809 15:17:07.830003 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 31976
I20260809 15:17:07.848820 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:07.847865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:37409 (local address 127.15.11.131:35071)
0809 15:17:07.848426 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:07.848556 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":273,"threads_started":1}
I20260809 15:17:07.850454 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:07.831083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.15.11.129:35641)
0809 15:17:07.831966 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:07.831977 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:07.832008 (+ 31us) server_negotiation.cc:407] Connection header received
0809 15:17:07.832093 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:07.832103 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:07.832467 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:07.832639 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:07.834746 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:07.835675 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:07.837076 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:07.837661 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:07.837755 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:07.844267 (+ 6512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:07.844283 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:07.844297 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:07.844333 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:07.847371 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:07.848282 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:07.848293 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:07.848296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:07.848476 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:07.848916 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:07.848921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:07.848924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:07.849373 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:07.849641 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:07.849921 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 15:17:07.850155 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":540,"threads_started":1}
I20260809 15:17:07.851755 32151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:07.935647 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 31976
I20260809 15:17:07.955469 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 32160
I20260809 15:17:07.968945 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:07.956342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56274 (local address 127.15.11.130:38913)
0809 15:17:07.956881 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:07.956886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:07.956908 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:17:07.956963 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:07.956967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:07.957026 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:07.957146 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:07.958696 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:07.959503 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:07.960806 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:07.961209 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:07.961301 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:07.963792 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:07.963797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:07.963800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:07.963830 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:07.966396 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:07.967017 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:07.967021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:07.967023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:07.967151 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:07.967487 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:07.967490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:07.967491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:07.967755 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:07.967961 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:07.968466 (+ 505us) server_negotiation.cc:299] Negotiation successful
0809 15:17:07.968656 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":280,"threads_started":1}
I20260809 15:17:07.969708 32292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:08.052186 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 32160
I20260809 15:17:08.075030 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 32302
I20260809 15:17:08.089428 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:08.075936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55382 (local address 127.15.11.131:45043)
0809 15:17:08.076525 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:08.076530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:08.076547 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:17:08.076603 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:08.076606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:08.076665 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:08.076768 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:08.078360 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:08.079126 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:08.080835 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:08.081224 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:08.081311 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:08.084058 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:08.084062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:08.084065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:08.084095 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:08.086813 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:08.087492 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:08.087495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:08.087497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:08.087626 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:08.087948 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:08.087951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:08.087953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:08.088381 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:08.088603 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:08.088882 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 15:17:08.089028 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":330,"threads_started":1}
I20260809 15:17:08.090016 32435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:08.171048 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 32302
2026-08-09T15:17:08Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24426 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:17:08 dist-test-slave-8w9v krb5kdc[32513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:17:08 dist-test-slave-8w9v krb5kdc[32513](info): set up 2 sockets
Aug 09 15:17:08 dist-test-slave-8w9v krb5kdc[32513](info): commencing operation
krb5kdc: starting...
W20260809 15:17:09.305068 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.007s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:17:09 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:17:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:17:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:17:09.565871 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39737
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:39737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:17:09.905889 32526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:09.906142 32526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:09.906227 32526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:09.912897 32526 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 15:17:09.912987 32526 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 15:17:09.913024 32526 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 15:17:09.913077 32526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:17:09.913129 32526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:09.913193 32526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:09.913237 32526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:17:09.913290 32526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:17:09.922633 32526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:39737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:09.923861 32526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:09.925144 32526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:09.936759 32531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:09.936730 32535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:09.936683 32533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:09.937639 32526 server_base.cc:1034] running on GCE node
I20260809 15:17:09.938411 32526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:09.939646 32526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:09.940881 32526 hybrid_clock.cc:648] HybridClock initialized: now 1786288629940817 us; error 24 us; skew 500 ppm
I20260809 15:17:09.941219 32526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:09 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288629, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:09.944485 32526 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:17:09.945894 32526 webserver.cc:460] Webserver started at http://127.15.11.190:42485/ using document root <none> and password file <none>
I20260809 15:17:09.946533 32526 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:09.946616 32526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:09.946862 32526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:17:09.949724 32526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "bdee2bb4a55442fea6968c6e47c46c70"
format_stamp: "Formatted at 2026-08-09 15:17:09 on dist-test-slave-8w9v"
server_key: "f6b9c22925f755a77a2e506e10858f31"
server_key_iv: "3425793b9cb49f0d34843bedd482ac26"
server_key_version: "encryptionkey@0"
I20260809 15:17:09.950488 32526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "bdee2bb4a55442fea6968c6e47c46c70"
format_stamp: "Formatted at 2026-08-09 15:17:09 on dist-test-slave-8w9v"
server_key: "f6b9c22925f755a77a2e506e10858f31"
server_key_iv: "3425793b9cb49f0d34843bedd482ac26"
server_key_version: "encryptionkey@0"
I20260809 15:17:09.955504 32526 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:17:09.958864 32542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:09.960104 32526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:17:09.960356 32526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "bdee2bb4a55442fea6968c6e47c46c70"
format_stamp: "Formatted at 2026-08-09 15:17:09 on dist-test-slave-8w9v"
server_key: "f6b9c22925f755a77a2e506e10858f31"
server_key_iv: "3425793b9cb49f0d34843bedd482ac26"
server_key_version: "encryptionkey@0"
I20260809 15:17:09.960588 32526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:09.977522 32526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:09.981748 32526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:09.982053 32526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:10.000397 32526 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:39737
I20260809 15:17:10.000367 32600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:39737 every 8 connection(s)
I20260809 15:17:10.001768 32526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:17:10.002521 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 32526
I20260809 15:17:10.002784 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:17:10.003608 15406 external_mini_cluster.cc:1428] Setting key dc93e8030fdd7f8d50047a443aafa51b
I20260809 15:17:10.006417 32601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:17:10 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:17:10.019141 32601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Bootstrap starting.
I20260809 15:17:10.023337 32601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Neither blocks nor log segments found. Creating new log.
I20260809 15:17:10.024605 32601 log.cc:824] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:10.027911 32601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: No bootstrap required, opened a new log
I20260809 15:17:10.028419 32604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:10.005674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41730 (local address 127.15.11.190:39737)
0809 15:17:10.006549 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:10.006564 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:10.006610 (+ 46us) server_negotiation.cc:407] Connection header received
0809 15:17:10.008955 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:10.008982 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:10.009500 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:10.009927 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:10.011211 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:10.013331 (+ 2120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:10.014632 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:10.015684 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:10.015861 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:10.019939 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:10.019949 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:10.019965 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:10.020006 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:10.024062 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:10.024753 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:10.024760 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:10.024768 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:10.024969 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:10.025425 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:10.025428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:10.025430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:10.025969 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:10.026243 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:10.026636 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 15:17:10.027084 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":357,"threads_started":1}
I20260809 15:17:10.035975 32601 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:10.036382 32601 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:17:10.036463 32601 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdee2bb4a55442fea6968c6e47c46c70, State: Initialized, Role: FOLLOWER
I20260809 15:17:10.036988 32601 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:10.037184 32601 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:17:10.037276 32601 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:17:10.037400 32601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:17:10.039383 32601 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:10.039803 32601 leader_election.cc:302] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdee2bb4a55442fea6968c6e47c46c70; no voters:
I20260809 15:17:10.040405 32601 leader_election.cc:288] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:17:10.040550 32606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:17:10.041849 32606 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 1 LEADER]: Becoming Leader. State: Replica: bdee2bb4a55442fea6968c6e47c46c70, State: Running, Role: LEADER
I20260809 15:17:10.042337 32606 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:10.042605 32601 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:17:10.045360 32608 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdee2bb4a55442fea6968c6e47c46c70. Latest consensus state: current_term: 1 leader_uuid: "bdee2bb4a55442fea6968c6e47c46c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } } }
I20260809 15:17:10.045336 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdee2bb4a55442fea6968c6e47c46c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } } }
I20260809 15:17:10.045770 32608 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: This master's current role is: LEADER
I20260809 15:17:10.045792 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: This master's current role is: LEADER
I20260809 15:17:10.048785 32615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:17:10.053944 32615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:17:10.062670 32615 catalog_manager.cc:1347] Generated new cluster ID: 4a4333384c404b418ab0c5e25a0730c7
I20260809 15:17:10.062774 32615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:17:10.078917 32615 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:17:10.080243 32615 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:17:10.090376 32615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Generated new TSK 0
I20260809 15:17:10.091161 32615 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:17:10.159562 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39737
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:17:10.497011 32628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:10.497316 32628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:10.497488 32628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:10.504325 32628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:10.504475 32628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:10.504666 32628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:17:10.513595 32628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:10.514748 32628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:10.515983 32628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:10.528726 32633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:10.528827 32634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:10.529276 32636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:10.529673 32628 server_base.cc:1034] running on GCE node
I20260809 15:17:10.530225 32628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:10.531070 32628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:10.532291 32628 hybrid_clock.cc:648] HybridClock initialized: now 1786288630532255 us; error 46 us; skew 500 ppm
I20260809 15:17:10.532603 32628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:10 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288630, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:10.535804 32628 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:17:10.537207 32628 webserver.cc:460] Webserver started at http://127.15.11.129:43875/ using document root <none> and password file <none>
I20260809 15:17:10.537835 32628 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:10.538059 32628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:10.538378 32628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:17:10.541280 32628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "a02e080e98014736aabafd5900943ac4"
format_stamp: "Formatted at 2026-08-09 15:17:10 on dist-test-slave-8w9v"
server_key: "855aa81242813d5bc63020cffbbc8771"
server_key_iv: "37d04c40f9dfc7ef924d483da1af4b66"
server_key_version: "encryptionkey@0"
I20260809 15:17:10.542063 32628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "a02e080e98014736aabafd5900943ac4"
format_stamp: "Formatted at 2026-08-09 15:17:10 on dist-test-slave-8w9v"
server_key: "855aa81242813d5bc63020cffbbc8771"
server_key_iv: "37d04c40f9dfc7ef924d483da1af4b66"
server_key_version: "encryptionkey@0"
I20260809 15:17:10.547268 32628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 15:17:10.550953 32643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:10.552217 32628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:17:10.552377 32628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "a02e080e98014736aabafd5900943ac4"
format_stamp: "Formatted at 2026-08-09 15:17:10 on dist-test-slave-8w9v"
server_key: "855aa81242813d5bc63020cffbbc8771"
server_key_iv: "37d04c40f9dfc7ef924d483da1af4b66"
server_key_version: "encryptionkey@0"
I20260809 15:17:10.552561 32628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:10.576112 32628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:10.579685 32628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:10.580000 32628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:10.580931 32628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:17:10.582254 32628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:17:10.582355 32628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:10.582496 32628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:17:10.582566 32628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:10.614928 32628 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:38481
I20260809 15:17:10.615015 304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:38481 every 8 connection(s)
I20260809 15:17:10.616252 32628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:17:10.626549 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 32628
I20260809 15:17:10.626768 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:17:10.627398 15406 external_mini_cluster.cc:1428] Setting key af70823868ab1771ec1a0ae5d196ad5b
Aug 09 15:17:10 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288630, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:17:10.636483 307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:10.618656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:42475 (local address 127.15.11.190:39737)
0809 15:17:10.619116 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:10.619120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:10.619886 (+ 766us) server_negotiation.cc:407] Connection header received
0809 15:17:10.621709 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:10.621714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:10.621816 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:10.621968 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:10.623442 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:10.624218 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:10.625688 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:10.626085 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:10.626219 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:10.630090 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:10.630098 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:10.630104 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:10.630150 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:10.633076 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:10.634087 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:10.634091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:10.634093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:10.634213 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:10.634708 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:10.634711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:10.634712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:10.634962 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:10.635079 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:10.636004 (+ 925us) server_negotiation.cc:299] Negotiation successful
0809 15:17:10.636241 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":248,"threads_started":1}
I20260809 15:17:10.638103 308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:10.619065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:42475)
0809 15:17:10.619714 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:10.619750 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:10.621442 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:10.622199 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:10.622207 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:10.622700 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:17:10.623265 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:10.623282 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:10.624369 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:10.624375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:10.625541 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:10.625553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:10.626314 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:10.626323 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:10.626566 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:10.627282 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:17:10.627304 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:17:10.629866 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:17:10.633246 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:10.633258 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:10.633277 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:10.633960 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:10.634320 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:10.634340 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:10.634346 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:10.634598 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:10.635096 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:17:10.635107 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:17:10.635580 (+ 473us) client_negotiation.cc:769] Sending connection context
0809 15:17:10.635915 (+ 335us) client_negotiation.cc:240] Negotiation successful
0809 15:17:10.636550 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":245,"threads_started":1}
I20260809 15:17:10.639669 305 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39737
I20260809 15:17:10.639989 305 heartbeater.cc:464] Registering TS with master...
I20260809 15:17:10.640738 305 heartbeater.cc:511] Master 127.15.11.190:39737 requested a full tablet report, sending...
I20260809 15:17:10.643152 32560 ts_manager.cc:194] Registered new tserver with Master: a02e080e98014736aabafd5900943ac4 (127.15.11.129:38481)
I20260809 15:17:10.644855 32560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:42475
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:17:10.691857 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39737
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:17:11.023159 311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:11.023471 311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:11.023641 311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:11.031035 311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:11.031179 311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:11.031352 311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:17:11.040882 311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:11.042126 311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:11.043488 311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:11.056488 321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:11.056440 319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:11.056413 317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:11.057623 311 server_base.cc:1034] running on GCE node
I20260809 15:17:11.058220 311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:11.059370 311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:11.060634 311 hybrid_clock.cc:648] HybridClock initialized: now 1786288631060574 us; error 52 us; skew 500 ppm
I20260809 15:17:11.060933 311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:11 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288631, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:11.064109 311 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:17:11.065522 311 webserver.cc:460] Webserver started at http://127.15.11.130:33099/ using document root <none> and password file <none>
I20260809 15:17:11.066171 311 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:11.066308 311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:11.066609 311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:17:11.069391 311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "b90b5679f1d74193bdcdfa82ddd24d91"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "f9ef2b9a8973ad2aa768fcc2740c224f"
server_key_iv: "4adbb7801be76edf48b4df50f80b163f"
server_key_version: "encryptionkey@0"
I20260809 15:17:11.070133 311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "b90b5679f1d74193bdcdfa82ddd24d91"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "f9ef2b9a8973ad2aa768fcc2740c224f"
server_key_iv: "4adbb7801be76edf48b4df50f80b163f"
server_key_version: "encryptionkey@0"
I20260809 15:17:11.075546 311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:17:11.078925 329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:11.080220 311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:17:11.080395 311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "b90b5679f1d74193bdcdfa82ddd24d91"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "f9ef2b9a8973ad2aa768fcc2740c224f"
server_key_iv: "4adbb7801be76edf48b4df50f80b163f"
server_key_version: "encryptionkey@0"
I20260809 15:17:11.080595 311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:11.093757 311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:11.097815 311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:11.098093 311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:11.099040 311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:17:11.100358 311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:17:11.100462 311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:11.100608 311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:17:11.100680 311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:11.133014 311 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:36485
I20260809 15:17:11.133139 452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:36485 every 8 connection(s)
I20260809 15:17:11.134279 311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:17:11.137542 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 311
I20260809 15:17:11.137760 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:17:11.138329 15406 external_mini_cluster.cc:1428] Setting key d3c501b0a35987008d42d6e85e260865
Aug 09 15:17:11 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288631, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:17:11.155790 307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.136812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:35955 (local address 127.15.11.190:39737)
0809 15:17:11.137133 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.137141 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.138286 (+ 1145us) server_negotiation.cc:407] Connection header received
0809 15:17:11.140182 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.140189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.140287 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.140422 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:11.141944 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.142733 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.144584 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.145072 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.145213 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.148981 (+ 3768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:11.148991 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:11.149001 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:11.149037 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:11.152516 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:11.153477 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:11.153481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:11.153483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:11.153613 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:11.154138 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:11.154141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:11.154143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:11.154378 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:11.154496 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.155398 (+ 902us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.155559 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:17:11.157506 456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.137297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:35955)
0809 15:17:11.138109 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:11.138153 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:11.139914 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:11.140662 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:11.140671 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:11.141201 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:17:11.141762 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.141781 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.142904 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.142909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.144426 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.144440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.145321 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.145330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.145581 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.146285 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:17:11.146309 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:17:11.148695 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:17:11.152681 (+ 3986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:11.152692 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:11.152711 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:11.153352 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:11.153730 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:11.153752 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:11.153759 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:11.154020 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:11.154516 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:17:11.154527 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:17:11.154991 (+ 464us) client_negotiation.cc:769] Sending connection context
0809 15:17:11.155333 (+ 342us) client_negotiation.cc:240] Negotiation successful
0809 15:17:11.155929 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":322,"threads_started":1}
I20260809 15:17:11.159111 454 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39737
I20260809 15:17:11.159410 454 heartbeater.cc:464] Registering TS with master...
I20260809 15:17:11.160159 454 heartbeater.cc:511] Master 127.15.11.190:39737 requested a full tablet report, sending...
I20260809 15:17:11.161597 32560 ts_manager.cc:194] Registered new tserver with Master: b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485)
I20260809 15:17:11.162392 32560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:35955
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:17:11.201129 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39737
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:17:11.529014 460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:11.529291 460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:11.529385 460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:11.536345 460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:11.536440 460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:11.536545 460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:17:11.545573 460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:39737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:11.546725 460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:11.548020 460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:11.561383 468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:11.561484 465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:11.561461 466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:11.562533 460 server_base.cc:1034] running on GCE node
I20260809 15:17:11.563076 460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:11.563961 460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:11.565153 460 hybrid_clock.cc:648] HybridClock initialized: now 1786288631565114 us; error 75 us; skew 500 ppm
I20260809 15:17:11.565424 460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:11 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288631, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:11.568398 460 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:17:11.569844 460 webserver.cc:460] Webserver started at http://127.15.11.131:32921/ using document root <none> and password file <none>
I20260809 15:17:11.570513 460 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:11.570612 460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:11.570852 460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:17:11.573668 460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "e91a48dd94454d64bafbe6831254927c"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "62aba2f6297aa6ae24a8e31266fce711"
server_key_iv: "b7221bb679c2c627aa6ae130f306c045"
server_key_version: "encryptionkey@0"
I20260809 15:17:11.574349 460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "e91a48dd94454d64bafbe6831254927c"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "62aba2f6297aa6ae24a8e31266fce711"
server_key_iv: "b7221bb679c2c627aa6ae130f306c045"
server_key_version: "encryptionkey@0"
I20260809 15:17:11.579254 460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 15:17:11.582667 475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:11.583976 460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:17:11.584232 460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "e91a48dd94454d64bafbe6831254927c"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "62aba2f6297aa6ae24a8e31266fce711"
server_key_iv: "b7221bb679c2c627aa6ae130f306c045"
server_key_version: "encryptionkey@0"
I20260809 15:17:11.584411 460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:11.599817 460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:11.603282 460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:11.603523 460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:11.604442 460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:17:11.605623 460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:17:11.605703 460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:11.605824 460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:17:11.605877 460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:11.638706 460 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:44495
I20260809 15:17:11.638744 596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:44495 every 8 connection(s)
I20260809 15:17:11.639997 460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:17:11.646665 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 460
I20260809 15:17:11.646886 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:17:11.647502 15406 external_mini_cluster.cc:1428] Setting key 488188dc03508c840e82c9384cd6cd3b
I20260809 15:17:11.647660 305 heartbeater.cc:503] Master 127.15.11.190:39737 was elected leader, sending a full tablet report...
Aug 09 15:17:11 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288631, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:17:11.661067 307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.642723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:34465 (local address 127.15.11.190:39737)
0809 15:17:11.642950 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.642956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.644317 (+ 1361us) server_negotiation.cc:407] Connection header received
0809 15:17:11.646151 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.646155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.646272 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.646361 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:11.648516 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.649656 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.651563 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.651941 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.652032 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.655140 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:11.655144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:11.655147 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:11.655174 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:11.657964 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:11.658969 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:11.658977 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:11.658982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:11.659154 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:11.659597 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:11.659600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:11.659602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:11.659853 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:11.659965 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.660649 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.660803 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:17:11.662022 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.643266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:34465)
0809 15:17:11.644102 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:11.644201 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:11.645873 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:11.646665 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:11.646677 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:11.647469 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:17:11.648348 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.648387 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.649832 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.649836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.651441 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.651453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.652162 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.652169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.652393 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.652968 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:17:11.652988 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:17:11.654970 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:17:11.658141 (+ 3171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:11.658164 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:11.658183 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:11.658815 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:11.659287 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:11.659292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:11.659296 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:11.659489 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:11.659959 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:17:11.659965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:17:11.660320 (+ 355us) client_negotiation.cc:769] Sending connection context
0809 15:17:11.660544 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 15:17:11.660897 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":340,"threads_started":1}
I20260809 15:17:11.663532 597 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39737
I20260809 15:17:11.663775 597 heartbeater.cc:464] Registering TS with master...
I20260809 15:17:11.664506 597 heartbeater.cc:511] Master 127.15.11.190:39737 requested a full tablet report, sending...
I20260809 15:17:11.665827 32560 ts_manager.cc:194] Registered new tserver with Master: e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495)
I20260809 15:17:11.666627 32560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:34465
I20260809 15:17:11.675083 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:17:11.692319 307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.679526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41732 (local address 127.15.11.190:39737)
0809 15:17:11.679721 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.679729 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.680069 (+ 340us) server_negotiation.cc:407] Connection header received
0809 15:17:11.680335 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.680341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.680431 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.680527 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:11.682010 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.682738 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.684341 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.684817 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.684929 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.687467 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:11.687471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:11.687474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:11.687505 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:11.690124 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:11.690693 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:11.690697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:11.690699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:11.690821 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:11.691120 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:11.691123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:11.691124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:11.691430 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:11.691616 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.691847 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.692012 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:17:11.696111 32560 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:17:11.702890 32560 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:17:11.726292 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.715164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:48116)
0809 15:17:11.716215 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:11.716231 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:11.716339 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:11.717658 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:11.717666 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:11.717697 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:11.718267 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.718281 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.720497 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.720502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.722954 (+ 2452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.722968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.724933 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.724941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.725162 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.725216 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 15:17:11.725675 (+ 459us) client_negotiation.cc:240] Negotiation successful
0809 15:17:11.725867 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"spinlock_wait_cycles":9088,"threads_started":1}
I20260809 15:17:11.727330 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.715963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48116 (local address 127.15.11.129:38481)
0809 15:17:11.716993 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.717004 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.717036 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:17:11.717151 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.717159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.717402 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.718513 (+ 1111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:11.718742 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.720338 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.723161 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.724674 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.726568 (+ 1894us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.726796 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.726869 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.727028 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":829,"thread_start_us":650,"threads_started":1}
I20260809 15:17:11.727367 611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.714462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:44495 (local address 127.0.0.1:44460)
0809 15:17:11.715227 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:11.715273 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:11.715439 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:11.717883 (+ 2444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:11.717887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:11.717922 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:11.718383 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.718405 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.721584 (+ 3179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.721591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.723752 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.723767 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.726577 (+ 2810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.726583 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.726760 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.726796 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 15:17:11.726862 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 15:17:11.726980 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"spinlock_wait_cycles":7680,"thread_start_us":361,"threads_started":1}
I20260809 15:17:11.727967 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.714794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.15.11.131:44495)
0809 15:17:11.717300 (+ 2506us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.717307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.717337 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:17:11.717441 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.717449 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.717669 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.718569 (+ 900us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:11.719849 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.721426 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.724958 (+ 3532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.726338 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.727143 (+ 805us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.727381 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.727445 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.727587 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2356,"thread_start_us":2185,"threads_started":1}
I20260809 15:17:11.731011 32697 tablet_service.cc:1467] Processing CreateTablet for tablet f89559cdb43546e59573460e82de63d4 (DEFAULT_TABLE table=test-table [id=938d26a01dea4f4cad9d1e5fdc9c25f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:17:11.731020 529 tablet_service.cc:1467] Processing CreateTablet for tablet f89559cdb43546e59573460e82de63d4 (DEFAULT_TABLE table=test-table [id=938d26a01dea4f4cad9d1e5fdc9c25f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:17:11.732309 32697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89559cdb43546e59573460e82de63d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:17:11.732373 529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89559cdb43546e59573460e82de63d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:17:11.738101 613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.715602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:36485 (local address 127.0.0.1:36770)
0809 15:17:11.727009 (+ 11407us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:11.727029 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:11.727153 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:11.727875 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:11.727879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:11.727914 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:11.728245 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.728257 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.733232 (+ 4975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.733239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.735287 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.735303 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.737315 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.737321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.737498 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.737531 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 15:17:11.737632 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 15:17:11.737764 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10667,"thread_start_us":346,"threads_started":1}
I20260809 15:17:11.738176 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.715934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36770 (local address 127.15.11.130:36485)
0809 15:17:11.727023 (+ 11089us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.727032 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.727273 (+ 241us) server_negotiation.cc:407] Connection header received
0809 15:17:11.727400 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.727408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.727613 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.727795 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:11.728468 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.733045 (+ 4577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.735470 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.736937 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.737075 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.737260 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.737701 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.737867 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10941,"thread_start_us":10790,"threads_started":1}
I20260809 15:17:11.741330 382 tablet_service.cc:1467] Processing CreateTablet for tablet f89559cdb43546e59573460e82de63d4 (DEFAULT_TABLE table=test-table [id=938d26a01dea4f4cad9d1e5fdc9c25f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:17:11.742415 382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f89559cdb43546e59573460e82de63d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:17:11.756258 618 tablet_bootstrap.cc:492] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: Bootstrap starting.
I20260809 15:17:11.756443 617 tablet_bootstrap.cc:492] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: Bootstrap starting.
I20260809 15:17:11.758136 619 tablet_bootstrap.cc:492] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: Bootstrap starting.
I20260809 15:17:11.760218 618 tablet_bootstrap.cc:654] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: Neither blocks nor log segments found. Creating new log.
I20260809 15:17:11.760789 617 tablet_bootstrap.cc:654] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: Neither blocks nor log segments found. Creating new log.
I20260809 15:17:11.761425 619 tablet_bootstrap.cc:654] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: Neither blocks nor log segments found. Creating new log.
I20260809 15:17:11.761663 618 log.cc:824] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:11.762043 617 log.cc:824] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:11.762455 619 log.cc:824] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:11.765372 619 tablet_bootstrap.cc:492] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: No bootstrap required, opened a new log
I20260809 15:17:11.765630 619 ts_tablet_manager.cc:1397] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 15:17:11.766273 617 tablet_bootstrap.cc:492] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: No bootstrap required, opened a new log
I20260809 15:17:11.766470 618 tablet_bootstrap.cc:492] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: No bootstrap required, opened a new log
I20260809 15:17:11.766536 617 ts_tablet_manager.cc:1397] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 15:17:11.766738 618 ts_tablet_manager.cc:1397] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 15:17:11.772642 619 raft_consensus.cc:348] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:11.773080 619 raft_consensus.cc:374] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:17:11.773211 619 raft_consensus.cc:729] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b90b5679f1d74193bdcdfa82ddd24d91, State: Initialized, Role: FOLLOWER
I20260809 15:17:11.773837 619 consensus_queue.cc:260] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:11.776952 454 heartbeater.cc:503] Master 127.15.11.190:39737 was elected leader, sending a full tablet report...
I20260809 15:17:11.777750 619 ts_tablet_manager.cc:1428] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: Time spent starting tablet: real 0.012s user 0.002s sys 0.011s
I20260809 15:17:11.778136 617 raft_consensus.cc:348] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:11.778604 617 raft_consensus.cc:374] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:17:11.778461 618 raft_consensus.cc:348] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:11.778733 617 raft_consensus.cc:729] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91a48dd94454d64bafbe6831254927c, State: Initialized, Role: FOLLOWER
I20260809 15:17:11.778934 618 raft_consensus.cc:374] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:17:11.779069 618 raft_consensus.cc:729] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02e080e98014736aabafd5900943ac4, State: Initialized, Role: FOLLOWER
I20260809 15:17:11.779359 617 consensus_queue.cc:260] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:11.779636 618 consensus_queue.cc:260] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:11.780745 597 heartbeater.cc:503] Master 127.15.11.190:39737 was elected leader, sending a full tablet report...
I20260809 15:17:11.781893 617 ts_tablet_manager.cc:1428] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260809 15:17:11.783921 618 ts_tablet_manager.cc:1428] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: Time spent starting tablet: real 0.017s user 0.005s sys 0.013s
I20260809 15:17:11.829981 626 raft_consensus.cc:482] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:17:11.830250 626 raft_consensus.cc:504] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:11.831787 626 leader_election.cc:288] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495), b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485)
I20260809 15:17:11.839825 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.832192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:42883 (local address 127.15.11.131:44495)
0809 15:17:11.832457 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.832463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.832786 (+ 323us) server_negotiation.cc:407] Connection header received
0809 15:17:11.833018 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.833026 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.833123 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.833218 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:11.833917 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.834971 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.837531 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.838698 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.838843 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.838914 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.839461 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.839581 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:17:11.839825 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.832944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:57485 (local address 127.15.11.130:36485)
0809 15:17:11.833173 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.833181 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.833539 (+ 358us) server_negotiation.cc:407] Connection header received
0809 15:17:11.833685 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.833689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.833744 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.833814 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:11.834309 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.835244 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.837532 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.838688 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.838844 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.838914 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.839457 (+ 543us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.839581 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:17:11.840044 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.832785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:36485 (local address 127.15.11.129:57485)
0809 15:17:11.833420 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:11.833437 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:11.833563 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:11.833856 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:11.833861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:11.833899 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:11.834159 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.834165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.835364 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.835369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.837396 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.837404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.838972 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.838979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.839233 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.839323 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 15:17:11.839412 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 15:17:11.839603 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":115,"thread_start_us":292,"threads_started":1}
I20260809 15:17:11.840162 627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.832038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:44495 (local address 127.15.11.129:42883)
0809 15:17:11.832630 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:11.832646 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:11.832837 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:11.833273 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:11.833277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:11.833303 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:11.833746 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.833757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.835098 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.835102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.837368 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:11.837384 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.838988 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.838992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:11.839217 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.839319 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 15:17:11.839415 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:17:11.839603 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":108,"thread_start_us":339,"threads_started":1}
I20260809 15:17:11.841070 403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89559cdb43546e59573460e82de63d4" candidate_uuid: "a02e080e98014736aabafd5900943ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" is_pre_election: true
I20260809 15:17:11.841070 550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89559cdb43546e59573460e82de63d4" candidate_uuid: "a02e080e98014736aabafd5900943ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91a48dd94454d64bafbe6831254927c" is_pre_election: true
I20260809 15:17:11.841550 403 raft_consensus.cc:2454] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02e080e98014736aabafd5900943ac4 in term 0.
I20260809 15:17:11.841557 550 raft_consensus.cc:2454] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02e080e98014736aabafd5900943ac4 in term 0.
I20260809 15:17:11.842228 32649 leader_election.cc:302] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a02e080e98014736aabafd5900943ac4, b90b5679f1d74193bdcdfa82ddd24d91; no voters:
I20260809 15:17:11.842583 626 raft_consensus.cc:2781] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:17:11.842741 626 raft_consensus.cc:482] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:17:11.842869 626 raft_consensus.cc:3037] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:17:11.845203 626 raft_consensus.cc:504] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:11.845748 626 leader_election.cc:288] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 1 election: Requested vote from peers e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495), b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485)
I20260809 15:17:11.846200 550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89559cdb43546e59573460e82de63d4" candidate_uuid: "a02e080e98014736aabafd5900943ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91a48dd94454d64bafbe6831254927c"
I20260809 15:17:11.846276 403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f89559cdb43546e59573460e82de63d4" candidate_uuid: "a02e080e98014736aabafd5900943ac4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b90b5679f1d74193bdcdfa82ddd24d91"
I20260809 15:17:11.846383 550 raft_consensus.cc:3037] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:17:11.846501 403 raft_consensus.cc:3037] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:17:11.848762 550 raft_consensus.cc:2454] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02e080e98014736aabafd5900943ac4 in term 1.
I20260809 15:17:11.848886 403 raft_consensus.cc:2454] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02e080e98014736aabafd5900943ac4 in term 1.
I20260809 15:17:11.849170 32649 leader_election.cc:302] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a02e080e98014736aabafd5900943ac4, e91a48dd94454d64bafbe6831254927c; no voters:
I20260809 15:17:11.849458 626 raft_consensus.cc:2781] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:17:11.849901 626 raft_consensus.cc:686] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 1 LEADER]: Becoming Leader. State: Replica: a02e080e98014736aabafd5900943ac4, State: Running, Role: LEADER
I20260809 15:17:11.850394 626 consensus_queue.cc:237] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:11.854887 32559 catalog_manager.cc:5495] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a02e080e98014736aabafd5900943ac4 (127.15.11.129). New cstate: current_term: 1 leader_uuid: "a02e080e98014736aabafd5900943ac4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } health_report { overall_health: HEALTHY } } }
W20260809 15:17:11.868520 306 tablet.cc:2369] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:17:11.886217 455 tablet.cc:2369] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:17:11.886536 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.880843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48128 (local address 127.15.11.129:38481)
0809 15:17:11.881082 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.881088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.881211 (+ 123us) server_negotiation.cc:407] Connection header received
0809 15:17:11.881461 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.881465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.881540 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.881686 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:17:11.882067 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.882796 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.884244 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.884802 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.884936 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.885436 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:17:11.885951 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:17:11.886114 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.886178 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.886267 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 15:17:11.891728 599 tablet.cc:2369] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:17:11.895051 403 raft_consensus.cc:1261] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [term 1 FOLLOWER]: Refusing update from remote peer a02e080e98014736aabafd5900943ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:17:11.895051 550 raft_consensus.cc:1261] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [term 1 FOLLOWER]: Refusing update from remote peer a02e080e98014736aabafd5900943ac4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:17:11.896018 629 consensus_queue.cc:1035] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:17:11.896420 626 consensus_queue.cc:1035] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:17:11.907202 633 mvcc.cc:205] Tried to move back new op lower bound from 7316638236231680000 to 7316638236063309824. Current Snapshot: MvccSnapshot[applied={T|T < 7316638236231680000}]
I20260809 15:17:11.911896 636 mvcc.cc:205] Tried to move back new op lower bound from 7316638236231680000 to 7316638236063309824. Current Snapshot: MvccSnapshot[applied={T|T < 7316638236231680000}]
I20260809 15:17:11.926433 32560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:17:11.926707 32560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:17:11.930023 32560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Sending DeleteTablet for 3 replicas of tablet f89559cdb43546e59573460e82de63d4
I20260809 15:17:11.931237 32697 tablet_service.cc:1514] Processing DeleteTablet for tablet f89559cdb43546e59573460e82de63d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:17:11 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:48116
I20260809 15:17:11.931351 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 32628
I20260809 15:17:11.931525 382 tablet_service.cc:1514] Processing DeleteTablet for tablet f89559cdb43546e59573460e82de63d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:17:11 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:36770
I20260809 15:17:11.932303 529 tablet_service.cc:1514] Processing DeleteTablet for tablet f89559cdb43546e59573460e82de63d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:17:11 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:44460
I20260809 15:17:11.944640 650 tablet_replica.cc:330] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: stopping tablet replica
I20260809 15:17:11.945096 650 raft_consensus.cc:2229] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:17:11.945627 650 raft_consensus.cc:2258] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:17:11.947606 646 tablet_replica.cc:330] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: stopping tablet replica
I20260809 15:17:11.947926 648 tablet_replica.cc:330] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: stopping tablet replica
I20260809 15:17:11.948006 646 raft_consensus.cc:2229] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:17:11.948418 648 raft_consensus.cc:2229] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:17:11.948939 648 raft_consensus.cc:2258] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:17:11.950124 650 ts_tablet_manager.cc:1905] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:17:11.951012 648 ts_tablet_manager.cc:1905] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:17:11.952066 646 raft_consensus.cc:2258] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:17:11.955114 646 ts_tablet_manager.cc:1905] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:17:11.957544 650 ts_tablet_manager.cc:1918] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:17:11.957743 650 log.cc:1196] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/f89559cdb43546e59573460e82de63d4
I20260809 15:17:11.958073 648 ts_tablet_manager.cc:1918] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:17:11.958288 648 log.cc:1196] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/f89559cdb43546e59573460e82de63d4
I20260809 15:17:11.958379 650 ts_tablet_manager.cc:1939] T f89559cdb43546e59573460e82de63d4 P e91a48dd94454d64bafbe6831254927c: Deleting consensus metadata
I20260809 15:17:11.958845 648 ts_tablet_manager.cc:1939] T f89559cdb43546e59573460e82de63d4 P b90b5679f1d74193bdcdfa82ddd24d91: Deleting consensus metadata
I20260809 15:17:11.960341 32546 catalog_manager.cc:4841] TS e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495): tablet f89559cdb43546e59573460e82de63d4 (table test-table [id=938d26a01dea4f4cad9d1e5fdc9c25f1]) successfully deleted
I20260809 15:17:11.960714 32546 catalog_manager.cc:4841] TS b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485): tablet f89559cdb43546e59573460e82de63d4 (table test-table [id=938d26a01dea4f4cad9d1e5fdc9c25f1]) successfully deleted
Aug 09 15:17:11 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:17:11.966121 646 ts_tablet_manager.cc:1918] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:17:11.966344 646 log.cc:1196] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/f89559cdb43546e59573460e82de63d4
I20260809 15:17:11.966884 646 ts_tablet_manager.cc:1939] T f89559cdb43546e59573460e82de63d4 P a02e080e98014736aabafd5900943ac4: Deleting consensus metadata
I20260809 15:17:11.968429 32545 catalog_manager.cc:4841] TS a02e080e98014736aabafd5900943ac4 (127.15.11.129:38481): tablet f89559cdb43546e59573460e82de63d4 (table test-table [id=938d26a01dea4f4cad9d1e5fdc9c25f1]) successfully deleted
I20260809 15:17:11.971334 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:11.932936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48134 (local address 127.15.11.129:38481)
0809 15:17:11.933239 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:11.933246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:11.942612 (+ 9366us) server_negotiation.cc:407] Connection header received
0809 15:17:11.943051 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:11.943057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:11.943132 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:11.943200 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:11.948269 (+ 5069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.949540 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.952526 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:11.953197 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:11.953294 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:11.963652 (+ 10358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:11.963671 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:11.963689 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:11.963754 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:11.968270 (+ 4516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:11.969167 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:11.969176 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:11.969179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:11.969361 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:11.969723 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:11.969728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:11.969731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:11.970167 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:11.970498 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:11.970728 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 15:17:11.970974 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260809 15:17:11.971976 32766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:12.076799 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 32628
I20260809 15:17:12.100196 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 311
Aug 09 15:17:12 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:17:12.115394 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:12.101073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36786 (local address 127.15.11.130:36485)
0809 15:17:12.101244 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:12.101248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:12.101267 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:17:12.101408 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:12.101412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:12.101484 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:12.101564 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:12.102921 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:12.103681 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:12.105299 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:12.105677 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:12.105789 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:12.110048 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:12.110061 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:12.110077 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:12.110110 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:12.112849 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:12.113534 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:12.113540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:12.113543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:12.113664 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:12.113980 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:12.113983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:12.113985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:12.114381 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:12.114610 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:12.114864 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 15:17:12.115040 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:17:12.116061 448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:12.207988 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 311
I20260809 15:17:12.228940 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 460
Aug 09 15:17:12 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:17:12.244092 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:12.229711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.15.11.131:44495)
0809 15:17:12.229905 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:12.229913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:12.229934 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:17:12.230165 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:12.230170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:12.230232 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:12.230352 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:12.231689 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:12.232478 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:12.233833 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:12.234235 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:12.234374 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:12.238814 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:12.238831 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:12.238847 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:12.238884 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:12.241534 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:12.242238 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:12.242245 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:12.242247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:12.242399 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:12.242741 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:12.242744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:12.242746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:12.243109 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:12.243321 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:12.243637 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 15:17:12.243809 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:17:12.244748 592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:12.332379 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 460
I20260809 15:17:12.354413 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 32526
I20260809 15:17:12.355615 32593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:12.465973 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 32526
I20260809 15:17:32.493768 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39737
--webserver_interface=127.15.11.190
--webserver_port=42485
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:39737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:17:32.840986 655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:32.841291 655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:32.841462 655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:32.848328 655 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 15:17:32.848445 655 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 15:17:32.848533 655 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 15:17:32.848603 655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:17:32.848677 655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:32.848770 655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:32.848905 655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:17:32.848985 655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:17:32.858614 655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:39737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=42485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:32.859860 655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:32.861315 655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:32.886785 666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:32.886749 664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:32.886829 663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:32.887137 655 server_base.cc:1034] running on GCE node
I20260809 15:17:32.888435 655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:32.890057 655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:32.891302 655 hybrid_clock.cc:648] HybridClock initialized: now 1786288652891226 us; error 29 us; skew 500 ppm
I20260809 15:17:32.891726 655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:32 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288652, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:32.895598 655 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:17:32.897397 655 webserver.cc:460] Webserver started at http://127.15.11.190:42485/ using document root <none> and password file <none>
I20260809 15:17:32.898156 655 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:32.898276 655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:32.904290 655 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 15:17:32.907243 673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:32.908730 655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:17:32.908928 655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "bdee2bb4a55442fea6968c6e47c46c70"
format_stamp: "Formatted at 2026-08-09 15:17:09 on dist-test-slave-8w9v"
server_key: "f6b9c22925f755a77a2e506e10858f31"
server_key_iv: "3425793b9cb49f0d34843bedd482ac26"
server_key_version: "encryptionkey@0"
I20260809 15:17:32.909998 655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:32.925042 655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:32.929353 655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:32.929646 655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:32.948355 655 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:39737
I20260809 15:17:32.948388 726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:39737 every 8 connection(s)
I20260809 15:17:32.949844 655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:17:32.952396 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 655
I20260809 15:17:32.953416 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:38481
--local_ip_for_outbound_sockets=127.15.11.129
--tserver_master_addrs=127.15.11.190:39737
--webserver_port=43875
--webserver_interface=127.15.11.129
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:17:32.959223 728 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:17:32.964412 728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Bootstrap starting.
I20260809 15:17:32.984709 728 log.cc:824] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:32.997254 728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:17:32.997869 728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Bootstrap complete.
I20260809 15:17:33.007236 728 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:33.008080 728 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdee2bb4a55442fea6968c6e47c46c70, State: Initialized, Role: FOLLOWER
I20260809 15:17:33.008767 728 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:33.009018 728 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:17:33.009140 728 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:17:33.009332 728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:17:33.013752 728 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:33.014379 728 leader_election.cc:302] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdee2bb4a55442fea6968c6e47c46c70; no voters:
I20260809 15:17:33.015062 728 leader_election.cc:288] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:17:33.015205 733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:17:33.017822 728 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:17:33.022738 733 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 2 LEADER]: Becoming Leader. State: Replica: bdee2bb4a55442fea6968c6e47c46c70, State: Running, Role: LEADER
I20260809 15:17:33.023459 733 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:33.026522 734 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bdee2bb4a55442fea6968c6e47c46c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } } }
I20260809 15:17:33.026600 735 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdee2bb4a55442fea6968c6e47c46c70. Latest consensus state: current_term: 2 leader_uuid: "bdee2bb4a55442fea6968c6e47c46c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } } }
I20260809 15:17:33.026937 734 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: This master's current role is: LEADER
I20260809 15:17:33.026964 735 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: This master's current role is: LEADER
I20260809 15:17:33.041258 751 catalog_manager.cc:1259] Loaded cluster ID: 4a4333384c404b418ab0c5e25a0730c7
I20260809 15:17:33.041368 751 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: loading cluster ID for follower catalog manager: success
I20260809 15:17:33.045421 751 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: acquiring CA information for follower catalog manager: success
I20260809 15:17:33.048348 751 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:17:33.048559 750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:17:33.051710 750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:17:33.052879 750 catalog_manager.cc:1259] Loaded cluster ID: 4a4333384c404b418ab0c5e25a0730c7
I20260809 15:17:33.052975 750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:17:33.054658 750 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:17:33.055913 750 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Loaded TSK: 0
I20260809 15:17:33.096040 750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Generated new TSK 1
I20260809 15:17:33.096459 750 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:17:33.328919 730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:33.329254 730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:33.329418 730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:33.336633 730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:33.336781 730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:33.336947 730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:17:33.346911 730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:38481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=43875
--tserver_master_addrs=127.15.11.190:39737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:33.348295 730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:33.349670 730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:33.365224 760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:33.365479 758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:33.365252 762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:33.366544 730 server_base.cc:1034] running on GCE node
I20260809 15:17:33.367265 730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:33.368382 730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:33.369626 730 hybrid_clock.cc:648] HybridClock initialized: now 1786288653369574 us; error 44 us; skew 500 ppm
I20260809 15:17:33.369972 730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:33 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288653, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:33.373380 730 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:17:33.375043 730 webserver.cc:460] Webserver started at http://127.15.11.129:43875/ using document root <none> and password file <none>
I20260809 15:17:33.375748 730 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:33.375919 730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:33.381744 730 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 15:17:33.384690 769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:33.386197 730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:17:33.386376 730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "a02e080e98014736aabafd5900943ac4"
format_stamp: "Formatted at 2026-08-09 15:17:10 on dist-test-slave-8w9v"
server_key: "855aa81242813d5bc63020cffbbc8771"
server_key_iv: "37d04c40f9dfc7ef924d483da1af4b66"
server_key_version: "encryptionkey@0"
I20260809 15:17:33.387365 730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:33.405251 730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:33.409962 730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:33.410243 730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:33.411239 730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:17:33.412616 730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:17:33.412738 730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:33.412861 730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:17:33.412947 730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:33.459715 730 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:38481
I20260809 15:17:33.459766 894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:38481 every 8 connection(s)
I20260809 15:17:33.461254 730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:17:33.466311 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 730
I20260809 15:17:33.467147 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:36485
--local_ip_for_outbound_sockets=127.15.11.130
--tserver_master_addrs=127.15.11.190:39737
--webserver_port=33099
--webserver_interface=127.15.11.130
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:17:33 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288653, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:17:33.488969 898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:33.464425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:57387)
0809 15:17:33.465263 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:33.465322 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:33.467945 (+ 2623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:33.469381 (+ 1436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:33.469392 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:33.470144 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:17:33.470884 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:33.470922 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:33.472641 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:33.472648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:33.474762 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:33.474779 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:33.475773 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:33.475782 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:33.476152 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:33.476873 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:17:33.476916 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:17:33.480276 (+ 3360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:17:33.485013 (+ 4737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:33.485020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:33.485030 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:33.485401 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:33.485869 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:33.485872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:33.485874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:33.486052 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:33.487045 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:17:33.487050 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:17:33.487321 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 15:17:33.487550 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 15:17:33.487929 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":355,"threads_started":1}
I20260809 15:17:33.489554 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:33.464274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:57387 (local address 127.15.11.190:39737)
0809 15:17:33.464906 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:33.464918 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:33.465445 (+ 527us) server_negotiation.cc:407] Connection header received
0809 15:17:33.468557 (+ 3112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:33.468570 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:33.468888 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:33.469169 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:33.471097 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:33.472466 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:33.474926 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:33.475537 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:33.475663 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:33.480548 (+ 4885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:33.480559 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:33.480578 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:33.480631 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:33.484851 (+ 4220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:33.485531 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:33.485542 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:33.485552 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:33.485759 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:33.486158 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:33.486165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:33.486170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:33.486903 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:33.487098 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:33.487835 (+ 737us) server_negotiation.cc:299] Negotiation successful
0809 15:17:33.488335 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":303,"threads_started":1}
I20260809 15:17:33.492215 895 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39737
I20260809 15:17:33.492506 895 heartbeater.cc:464] Registering TS with master...
I20260809 15:17:33.493214 895 heartbeater.cc:511] Master 127.15.11.190:39737 requested a full tablet report, sending...
I20260809 15:17:33.495431 691 ts_manager.cc:194] Registered new tserver with Master: a02e080e98014736aabafd5900943ac4 (127.15.11.129:38481)
I20260809 15:17:33.496934 691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:57387
W20260809 15:17:33.807842 899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:33.808107 899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:33.808275 899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:33.815368 899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:33.815495 899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:33.815686 899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:17:33.825661 899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:36485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=33099
--tserver_master_addrs=127.15.11.190:39737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:33.826891 899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:33.828248 899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:33.841439 905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:33.841462 908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:33.841408 906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:33.842231 899 server_base.cc:1034] running on GCE node
I20260809 15:17:33.842703 899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:33.843576 899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:33.844805 899 hybrid_clock.cc:648] HybridClock initialized: now 1786288653844738 us; error 38 us; skew 500 ppm
I20260809 15:17:33.845129 899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:33 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288653, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:33.848079 899 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:17:33.849478 899 webserver.cc:460] Webserver started at http://127.15.11.130:33099/ using document root <none> and password file <none>
I20260809 15:17:33.850080 899 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:33.850190 899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:33.855178 899 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:17:33.857752 915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:33.859052 899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:17:33.859227 899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "b90b5679f1d74193bdcdfa82ddd24d91"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "f9ef2b9a8973ad2aa768fcc2740c224f"
server_key_iv: "4adbb7801be76edf48b4df50f80b163f"
server_key_version: "encryptionkey@0"
I20260809 15:17:33.860248 899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:33.880722 899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:33.884828 899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:33.885087 899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:33.885998 899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:17:33.887262 899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:17:33.887390 899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:33.887512 899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:17:33.887617 899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:33.921206 899 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:36485
I20260809 15:17:33.921265 1037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:36485 every 8 connection(s)
I20260809 15:17:33.922749 899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:17:33.931403 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 899
I20260809 15:17:33.932003 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:44495
--local_ip_for_outbound_sockets=127.15.11.131
--tserver_master_addrs=127.15.11.190:39737
--webserver_port=32921
--webserver_interface=127.15.11.131
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:17:33 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288653, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:17:33.941754 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:33.925227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:34925 (local address 127.15.11.190:39737)
0809 15:17:33.925408 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:33.925412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:33.926571 (+ 1159us) server_negotiation.cc:407] Connection header received
0809 15:17:33.928464 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:33.928469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:33.928563 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:33.928678 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:33.930151 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:33.930846 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:33.932532 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:33.932952 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:33.933090 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:33.936447 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:33.936452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:33.936455 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:33.936486 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:33.938823 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:33.939517 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:33.939523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:33.939526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:33.939660 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:33.940070 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:33.940075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:33.940084 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:33.940409 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:33.940560 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:33.941179 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 15:17:33.941402 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:17:33.942636 1040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:33.925723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:34925)
0809 15:17:33.926410 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:33.926453 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:33.928109 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:33.928925 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:33.928933 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:33.929432 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:17:33.929972 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:33.929989 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:33.931027 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:33.931037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:33.932386 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:33.932397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:33.933202 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:33.933210 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:33.933430 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:33.934093 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:17:33.934118 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:17:33.936232 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:17:33.938972 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:33.938980 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:33.938994 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:33.939389 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:33.939768 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:33.939772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:33.939776 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:33.939954 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:33.940535 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:17:33.940542 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:17:33.940849 (+ 307us) client_negotiation.cc:769] Sending connection context
0809 15:17:33.941085 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 15:17:33.941550 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":271,"threads_started":1}
I20260809 15:17:33.943972 1038 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39737
I20260809 15:17:33.944290 1038 heartbeater.cc:464] Registering TS with master...
I20260809 15:17:33.945123 1038 heartbeater.cc:511] Master 127.15.11.190:39737 requested a full tablet report, sending...
I20260809 15:17:33.946640 691 ts_manager.cc:194] Registered new tserver with Master: b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485)
I20260809 15:17:33.947412 691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:34925
W20260809 15:17:34.268888 1041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:34.269212 1041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:34.269377 1041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:34.276326 1041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:34.276463 1041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:34.276662 1041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:17:34.286863 1041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:44495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=32921
--tserver_master_addrs=127.15.11.190:39737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:34.288080 1041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:34.289413 1041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:34.302316 1046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:34.302266 1047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:34.302353 1049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:34.303049 1041 server_base.cc:1034] running on GCE node
I20260809 15:17:34.303656 1041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:34.304579 1041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:34.305816 1041 hybrid_clock.cc:648] HybridClock initialized: now 1786288654305782 us; error 45 us; skew 500 ppm
I20260809 15:17:34.306126 1041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:34 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288654, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:34.309434 1041 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:17:34.310901 1041 webserver.cc:460] Webserver started at http://127.15.11.131:32921/ using document root <none> and password file <none>
I20260809 15:17:34.311566 1041 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:34.311714 1041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:34.317162 1041 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 15:17:34.319861 1057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:34.321166 1041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:17:34.321338 1041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "e91a48dd94454d64bafbe6831254927c"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "62aba2f6297aa6ae24a8e31266fce711"
server_key_iv: "b7221bb679c2c627aa6ae130f306c045"
server_key_version: "encryptionkey@0"
I20260809 15:17:34.322400 1041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:34.333411 1041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:34.337095 1041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:34.337373 1041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:34.338284 1041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:17:34.339553 1041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:17:34.339666 1041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:34.339789 1041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:17:34.339883 1041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:34.371732 1041 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:44495
I20260809 15:17:34.371820 1180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:44495 every 8 connection(s)
I20260809 15:17:34.373261 1041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:17:34.381425 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 1041
Aug 09 15:17:34 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288654, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:17:34.394033 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.375882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:42747 (local address 127.15.11.190:39737)
0809 15:17:34.376081 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.376086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.377275 (+ 1189us) server_negotiation.cc:407] Connection header received
0809 15:17:34.379174 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.379181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.379288 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.379415 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:34.381028 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.381819 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.383767 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.384234 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.384376 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.387923 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:34.387927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:34.387930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:34.387963 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:34.390824 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.391539 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.391544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.391549 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.391694 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.392120 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.392140 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.392143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.392521 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:34.392684 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.393346 (+ 662us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.393679 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:17:34.394836 1184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.376323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:42747)
0809 15:17:34.377067 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.377119 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:34.378896 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:34.379670 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:34.379677 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:34.380294 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:17:34.380845 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.380873 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.381992 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.381997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.383614 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.383627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.384456 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.384465 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.384691 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.385399 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:17:34.385420 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:17:34.387702 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:17:34.390976 (+ 3274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:34.390984 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:34.390998 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:34.391414 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:34.391818 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:34.391821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:34.391823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:34.392001 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:34.392664 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:17:34.392672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:17:34.392976 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 15:17:34.393235 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 15:17:34.393725 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":287,"threads_started":1}
I20260809 15:17:34.396427 1181 heartbeater.cc:347] Connected to a master server at 127.15.11.190:39737
I20260809 15:17:34.396725 1181 heartbeater.cc:464] Registering TS with master...
I20260809 15:17:34.396826 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.382344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.15.11.190:39737)
0809 15:17:34.383169 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.383177 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.383194 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:17:34.383248 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.383251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.383300 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.383420 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:34.385120 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.385921 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.387685 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.388196 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.388359 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.390990 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:34.390996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:34.391002 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:34.391033 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:34.393840 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.394616 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.394624 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.394629 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.394748 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.395209 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.395213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.395218 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.395478 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:34.395732 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.396077 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.396424 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":548,"threads_started":1}
I20260809 15:17:34.397334 1181 heartbeater.cc:511] Master 127.15.11.190:39737 requested a full tablet report, sending...
I20260809 15:17:34.398777 690 ts_manager.cc:194] Registered new tserver with Master: e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495)
I20260809 15:17:34.399847 690 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:42747
I20260809 15:17:34.411545 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:17:34.418542 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.412599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.15.11.190:39737)
0809 15:17:34.412859 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.412866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.413188 (+ 322us) server_negotiation.cc:407] Connection header received
0809 15:17:34.413402 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.413412 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.413565 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.413773 (+ 208us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:17:34.414164 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.414977 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.416533 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.417000 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.417102 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.417587 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:17:34.417907 (+ 320us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 15:17:34.418198 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51232: token expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 15:17:34.418687 1186 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51232: token expired
W20260809 15:17:34.419313 605 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.11.190:39737)
I20260809 15:17:34.432514 1186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.419781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.15.11.190:39737)
0809 15:17:34.419966 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.419972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.420016 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:17:34.420251 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.420257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.420317 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.420414 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:34.421914 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.422561 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.423886 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.424352 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.424485 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.427211 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:34.427216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:34.427220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:34.427250 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:34.429866 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.430510 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.430517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.430521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.430662 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.431054 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.431060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.431063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.431364 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:34.431622 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.431943 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.432184 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:17:34.435853 690 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:17:34.442998 690 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:17:34.478681 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.454068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:36485 (local address 127.0.0.1:52956)
0809 15:17:34.455130 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.455173 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:34.455342 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:34.468986 (+ 13644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:34.468997 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:34.469039 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:34.469566 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.469583 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.473025 (+ 3442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.473030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.475559 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.475574 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.477827 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.477836 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.478006 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.478039 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 15:17:34.478127 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:17:34.478251 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":792,"thread_start_us":636,"threads_started":1}
I20260809 15:17:34.478835 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.455950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.15.11.129:38481)
0809 15:17:34.470605 (+ 14655us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.470615 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.470648 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:17:34.470754 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.470763 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.470968 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.471178 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:34.472085 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.473717 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.476248 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.477210 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.477418 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.477658 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.478377 (+ 719us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.478580 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14505,"thread_start_us":14354,"threads_started":1}
I20260809 15:17:34.479723 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.455959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.15.11.131:44495)
0809 15:17:34.471084 (+ 15125us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.471094 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.471713 (+ 619us) server_negotiation.cc:407] Connection header received
0809 15:17:34.471890 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.471898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.472104 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.472314 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:34.472898 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.474337 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.476552 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.478429 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.478556 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.478756 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.479354 (+ 598us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.479474 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14944,"thread_start_us":14766,"threads_started":1}
I20260809 15:17:34.479719 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.454279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52956 (local address 127.15.11.130:36485)
0809 15:17:34.468236 (+ 13957us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.468248 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.468283 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:17:34.468394 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.468403 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.468691 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.471048 (+ 2357us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:34.471367 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.472880 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.475740 (+ 2860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.477311 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.477463 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.477664 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.479206 (+ 1542us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.479379 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13755,"thread_start_us":13619,"threads_started":1}
I20260809 15:17:34.479760 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.455525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:44495 (local address 127.0.0.1:43794)
0809 15:17:34.471573 (+ 16048us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.471591 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:34.471722 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:34.472324 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:34.472329 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:34.472356 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:34.472725 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.472737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.474493 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.474500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.476383 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.476398 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.478968 (+ 2570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.478977 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.479167 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.479197 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:17:34.479292 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 15:17:34.479407 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15920,"mutex_wait_us":18,"thread_start_us":15735,"threads_started":1}
I20260809 15:17:34.480464 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.454594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:45704)
0809 15:17:34.466115 (+ 11521us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.466143 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:34.466327 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:34.471490 (+ 5163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:34.471494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:34.471518 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:34.471884 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.471896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.473906 (+ 2010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.473911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.476055 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.476067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.477510 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.477518 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.477711 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.477767 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 15:17:34.477928 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 15:17:34.478102 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11335,"mutex_wait_us":38,"thread_start_us":11842,"threads_started":1}
I20260809 15:17:34.483695 1111 tablet_service.cc:1467] Processing CreateTablet for tablet 1dca3295f096472aad5bed1c145ac01e (DEFAULT_TABLE table=test-table [id=4ec07360ab1241baa7b57b1335d7aa31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:17:34.483695 824 tablet_service.cc:1467] Processing CreateTablet for tablet 1dca3295f096472aad5bed1c145ac01e (DEFAULT_TABLE table=test-table [id=4ec07360ab1241baa7b57b1335d7aa31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:17:34.483695 967 tablet_service.cc:1467] Processing CreateTablet for tablet 1dca3295f096472aad5bed1c145ac01e (DEFAULT_TABLE table=test-table [id=4ec07360ab1241baa7b57b1335d7aa31]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:17:34.485004 1111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dca3295f096472aad5bed1c145ac01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:17:34.485080 967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dca3295f096472aad5bed1c145ac01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:17:34.485105 824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dca3295f096472aad5bed1c145ac01e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:17:34.499289 1199 tablet_bootstrap.cc:492] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: Bootstrap starting.
I20260809 15:17:34.500069 895 heartbeater.cc:503] Master 127.15.11.190:39737 was elected leader, sending a full tablet report...
I20260809 15:17:34.502828 1199 tablet_bootstrap.cc:654] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: Neither blocks nor log segments found. Creating new log.
I20260809 15:17:34.503456 1198 tablet_bootstrap.cc:492] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: Bootstrap starting.
I20260809 15:17:34.504276 1199 log.cc:824] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:34.505470 1200 tablet_bootstrap.cc:492] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Bootstrap starting.
I20260809 15:17:34.506670 1198 tablet_bootstrap.cc:654] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: Neither blocks nor log segments found. Creating new log.
I20260809 15:17:34.507645 1198 log.cc:824] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:34.508262 1200 tablet_bootstrap.cc:654] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Neither blocks nor log segments found. Creating new log.
I20260809 15:17:34.509644 1200 log.cc:824] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:34.510588 1198 tablet_bootstrap.cc:492] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: No bootstrap required, opened a new log
I20260809 15:17:34.510893 1198 ts_tablet_manager.cc:1397] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260809 15:17:34.511231 1199 tablet_bootstrap.cc:492] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: No bootstrap required, opened a new log
I20260809 15:17:34.511503 1199 ts_tablet_manager.cc:1397] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260809 15:17:34.513295 1200 tablet_bootstrap.cc:492] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: No bootstrap required, opened a new log
I20260809 15:17:34.513540 1200 ts_tablet_manager.cc:1397] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260809 15:17:34.519416 1198 raft_consensus.cc:348] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:34.519824 1198 raft_consensus.cc:374] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:17:34.519948 1198 raft_consensus.cc:729] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91a48dd94454d64bafbe6831254927c, State: Initialized, Role: FOLLOWER
I20260809 15:17:34.520592 1198 consensus_queue.cc:260] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:34.522490 1199 raft_consensus.cc:348] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:34.522892 1199 raft_consensus.cc:374] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:17:34.522987 1199 raft_consensus.cc:729] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b90b5679f1d74193bdcdfa82ddd24d91, State: Initialized, Role: FOLLOWER
I20260809 15:17:34.523573 1199 consensus_queue.cc:260] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:34.524315 1200 raft_consensus.cc:348] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:34.524767 1200 raft_consensus.cc:374] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:17:34.524914 1200 raft_consensus.cc:729] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02e080e98014736aabafd5900943ac4, State: Initialized, Role: FOLLOWER
I20260809 15:17:34.524914 1038 heartbeater.cc:503] Master 127.15.11.190:39737 was elected leader, sending a full tablet report...
I20260809 15:17:34.525558 1199 ts_tablet_manager.cc:1428] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: Time spent starting tablet: real 0.014s user 0.013s sys 0.002s
I20260809 15:17:34.525676 1200 consensus_queue.cc:260] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:34.527460 1200 ts_tablet_manager.cc:1428] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260809 15:17:34.528368 1181 heartbeater.cc:503] Master 127.15.11.190:39737 was elected leader, sending a full tablet report...
I20260809 15:17:34.528949 1198 ts_tablet_manager.cc:1428] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: Time spent starting tablet: real 0.018s user 0.015s sys 0.002s
I20260809 15:17:34.617988 1206 raft_consensus.cc:482] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:17:34.618345 1206 raft_consensus.cc:504] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:34.620435 1206 leader_election.cc:288] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495), a02e080e98014736aabafd5900943ac4 (127.15.11.129:38481)
W20260809 15:17:34.624933 1182 tablet.cc:2369] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:17:34.629673 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.620940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:33241 (local address 127.15.11.129:38481)
0809 15:17:34.621133 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.621140 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.621874 (+ 734us) server_negotiation.cc:407] Connection header received
0809 15:17:34.622057 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.622062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.622128 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.622225 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:34.623073 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.624212 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.627312 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.628558 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.628720 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.628792 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.629298 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.629427 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:17:34.629666 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.620820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:38371 (local address 127.15.11.131:44495)
0809 15:17:34.621032 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.621039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.621855 (+ 816us) server_negotiation.cc:407] Connection header received
0809 15:17:34.622044 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.622049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.622128 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.622225 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:17:34.623073 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.624215 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.627313 (+ 3098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.628558 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.628720 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.628792 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.629298 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.629430 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:17:34.629974 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.620731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:44495 (local address 127.15.11.130:38371)
0809 15:17:34.621707 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.621727 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:34.621881 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:34.622252 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:34.622257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:34.622282 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:34.622905 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.622914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.624353 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.624357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.627136 (+ 2779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.627152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.628827 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.628832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.629060 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.629166 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 15:17:34.629257 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:17:34.629470 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":345,"thread_start_us":602,"threads_started":1}
I20260809 15:17:34.629998 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.620853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.15.11.130:33241)
0809 15:17:34.621762 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.621777 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:34.621932 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:34.622252 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:34.622257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:34.622350 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:17:34.622867 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.622886 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.624352 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.624356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.627136 (+ 2780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:34.627152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.628824 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.628831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:34.629057 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.629165 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 15:17:34.629257 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 15:17:34.629470 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":371,"spinlock_wait_cycles":37760,"thread_start_us":605,"threads_started":1}
I20260809 15:17:34.631201 1133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dca3295f096472aad5bed1c145ac01e" candidate_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91a48dd94454d64bafbe6831254927c" is_pre_election: true
I20260809 15:17:34.631201 846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dca3295f096472aad5bed1c145ac01e" candidate_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a02e080e98014736aabafd5900943ac4" is_pre_election: true
I20260809 15:17:34.631824 846 raft_consensus.cc:2454] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b90b5679f1d74193bdcdfa82ddd24d91 in term 0.
I20260809 15:17:34.631827 1133 raft_consensus.cc:2454] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b90b5679f1d74193bdcdfa82ddd24d91 in term 0.
I20260809 15:17:34.632550 918 leader_election.cc:302] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a02e080e98014736aabafd5900943ac4, b90b5679f1d74193bdcdfa82ddd24d91; no voters:
I20260809 15:17:34.632885 1206 raft_consensus.cc:2781] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:17:34.633061 1206 raft_consensus.cc:482] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:17:34.633174 1206 raft_consensus.cc:3037] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:17:34.635456 1206 raft_consensus.cc:504] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:34.636092 1206 leader_election.cc:288] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [CANDIDATE]: Term 1 election: Requested vote from peers e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495), a02e080e98014736aabafd5900943ac4 (127.15.11.129:38481)
I20260809 15:17:34.636508 1133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dca3295f096472aad5bed1c145ac01e" candidate_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e91a48dd94454d64bafbe6831254927c"
I20260809 15:17:34.636636 846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dca3295f096472aad5bed1c145ac01e" candidate_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a02e080e98014736aabafd5900943ac4"
I20260809 15:17:34.636727 1133 raft_consensus.cc:3037] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:17:34.636875 846 raft_consensus.cc:3037] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:17:34.638970 1133 raft_consensus.cc:2454] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b90b5679f1d74193bdcdfa82ddd24d91 in term 1.
I20260809 15:17:34.639067 846 raft_consensus.cc:2454] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b90b5679f1d74193bdcdfa82ddd24d91 in term 1.
I20260809 15:17:34.639410 919 leader_election.cc:302] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b90b5679f1d74193bdcdfa82ddd24d91, e91a48dd94454d64bafbe6831254927c; no voters:
I20260809 15:17:34.639745 1206 raft_consensus.cc:2781] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:17:34.640239 1206 raft_consensus.cc:686] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 1 LEADER]: Becoming Leader. State: Replica: b90b5679f1d74193bdcdfa82ddd24d91, State: Running, Role: LEADER
I20260809 15:17:34.640755 1206 consensus_queue.cc:237] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:34.645169 689 catalog_manager.cc:5495] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 reported cstate change: term changed from 0 to 1, leader changed from <none> to b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130). New cstate: current_term: 1 leader_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } health_report { overall_health: UNKNOWN } } }
I20260809 15:17:34.669776 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.664874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52972 (local address 127.15.11.130:36485)
0809 15:17:34.665078 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.665082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.665097 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:17:34.665245 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.665248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.665333 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.665458 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:17:34.665768 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.666455 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.667830 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.668284 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.668371 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.668855 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:17:34.669339 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:17:34.669439 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.669519 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.669578 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 15:17:34.674535 1039 tablet.cc:2369] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:17:34.677899 846 raft_consensus.cc:1261] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Refusing update from remote peer b90b5679f1d74193bdcdfa82ddd24d91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:17:34.677985 1133 raft_consensus.cc:1261] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [term 1 FOLLOWER]: Refusing update from remote peer b90b5679f1d74193bdcdfa82ddd24d91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:17:34.678947 1210 consensus_queue.cc:1035] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:17:34.679276 1206 consensus_queue.cc:1035] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:17:34.688359 1213 mvcc.cc:205] Tried to move back new op lower bound from 7316638329550708736 to 7316638329412734976. Current Snapshot: MvccSnapshot[applied={T|T < 7316638329550708736}]
I20260809 15:17:34.691672 1215 mvcc.cc:205] Tried to move back new op lower bound from 7316638329550708736 to 7316638329412734976. Current Snapshot: MvccSnapshot[applied={T|T < 7316638329550708736}]
I20260809 15:17:34.706874 689 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:17:34.707132 689 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:17:34.710537 689 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Sending DeleteTablet for 3 replicas of tablet 1dca3295f096472aad5bed1c145ac01e
I20260809 15:17:34.711650 967 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dca3295f096472aad5bed1c145ac01e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:17:34 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:52956
I20260809 15:17:34.711762 1111 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dca3295f096472aad5bed1c145ac01e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:17:34 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:43794
I20260809 15:17:34.712186 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 730
I20260809 15:17:34.713034 824 tablet_service.cc:1514] Processing DeleteTablet for tablet 1dca3295f096472aad5bed1c145ac01e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:17:34 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:45704
I20260809 15:17:34.724555 1224 tablet_replica.cc:330] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: stopping tablet replica
I20260809 15:17:34.725667 1224 raft_consensus.cc:2229] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:17:34.726794 1224 raft_consensus.cc:2258] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:17:34.729569 1224 ts_tablet_manager.cc:1905] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:17:34.738132 1224 ts_tablet_manager.cc:1918] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:17:34.740001 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.713257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45714 (local address 127.15.11.129:38481)
0809 15:17:34.713808 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.713817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.713839 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:17:34.713922 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.713930 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.714027 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.714139 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:34.718217 (+ 4078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.719811 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.722440 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.723352 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.723502 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.728875 (+ 5373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:34.728897 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:34.728919 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:34.728974 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:34.734173 (+ 5199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.735418 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.735432 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.735438 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.735640 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.736297 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.736305 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.736310 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.736937 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:34.737315 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.738959 (+ 1644us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.739279 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388}
I20260809 15:17:34.741374 1225 tablet_replica.cc:330] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: stopping tablet replica
I20260809 15:17:34.741415 890 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:34.741583 1223 tablet_replica.cc:330] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: stopping tablet replica
I20260809 15:17:34.741648 1224 log.cc:1196] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/1dca3295f096472aad5bed1c145ac01e
I20260809 15:17:34.742138 1223 raft_consensus.cc:2229] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:17:34.742379 1224 ts_tablet_manager.cc:1939] T 1dca3295f096472aad5bed1c145ac01e P e91a48dd94454d64bafbe6831254927c: Deleting consensus metadata
I20260809 15:17:34.743105 1223 raft_consensus.cc:2258] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:17:34.744434 678 catalog_manager.cc:4841] TS e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495): tablet 1dca3295f096472aad5bed1c145ac01e (table test-table [id=4ec07360ab1241baa7b57b1335d7aa31]) successfully deleted
I20260809 15:17:34.746290 1223 ts_tablet_manager.cc:1905] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:17:34.753971 1223 ts_tablet_manager.cc:1918] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:17:34.754140 1223 log.cc:1196] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/1dca3295f096472aad5bed1c145ac01e
I20260809 15:17:34.754778 1223 ts_tablet_manager.cc:1939] T 1dca3295f096472aad5bed1c145ac01e P b90b5679f1d74193bdcdfa82ddd24d91: Deleting consensus metadata
I20260809 15:17:34.756738 678 catalog_manager.cc:4841] TS b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485): tablet 1dca3295f096472aad5bed1c145ac01e (table test-table [id=4ec07360ab1241baa7b57b1335d7aa31]) successfully deleted
I20260809 15:17:34.744032 1225 raft_consensus.cc:2229] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:17:34.833760 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 730
W20260809 15:17:34.860625 677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.11.129:38481 (error 108)
I20260809 15:17:34.860939 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 899
I20260809 15:17:34.862977 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.862438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:45724)
0809 15:17:34.862679 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.862786 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
W20260809 15:17:34.863256 677 catalog_manager.cc:4584] TS a02e080e98014736aabafd5900943ac4 (127.15.11.129:38481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1dca3295f096472aad5bed1c145ac01e: Network error: Client connection negotiation failed: client connection to 127.15.11.129:38481: connect: Connection refused (error 111)
I20260809 15:17:34.878253 1193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.862440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.15.11.130:36485)
0809 15:17:34.862621 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:34.862627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:34.862645 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:17:34.862715 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:34.862720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:34.862800 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:34.862914 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:34.864826 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.865662 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.867523 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:34.867989 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:34.868078 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:34.871615 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:34.871630 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:34.871648 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:34.871683 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:34.874831 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.875665 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.875671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.875674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.875812 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:34.876483 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:34.876488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:34.876492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:34.876987 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:34.877383 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:34.877610 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 15:17:34.877837 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:17:34.879031 1033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:34.916998 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.916524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:45726)
0809 15:17:34.916728 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.916837 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 15:17:34.918113 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.917740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:45732)
0809 15:17:34.917898 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.918001 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 15:17:34.978670 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.978177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:45738)
0809 15:17:34.978388 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.978491 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 15:17:34.979763 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:34.979385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:45746)
0809 15:17:34.979562 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:34.979643 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 15:17:34.994812 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 899
I20260809 15:17:35.017944 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 1041
I20260809 15:17:35.032547 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:35.018756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.15.11.131:44495)
0809 15:17:35.018942 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:35.018949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:35.019003 (+ 54us) server_negotiation.cc:407] Connection header received
0809 15:17:35.019258 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:35.019263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:35.019331 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:35.019438 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:35.020942 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:35.021743 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:35.023111 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:35.023508 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:35.023647 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:35.026457 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:35.026472 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:35.026489 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:35.026538 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:35.029431 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:35.030178 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:35.030194 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:35.030202 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:35.030345 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:35.030752 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:35.030758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:35.030761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:35.031119 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:35.031437 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:35.031808 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 15:17:35.032050 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:17:35.033167 1175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:35.060535 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:35.059917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:45750)
0809 15:17:35.060209 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:35.060345 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260809 15:17:35.061676 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:35.061235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:45754)
0809 15:17:35.061452 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:35.061539 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 15:17:35.123771 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 1041
I20260809 15:17:35.145402 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 655
I20260809 15:17:35.146641 721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:17:35.261475 1194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:35.260600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:38481 (local address 127.0.0.1:45760)
0809 15:17:35.260835 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:35.260975 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 15:17:35.261509 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 655
I20260809 15:17:55.289171 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39737
--webserver_interface=127.15.11.190
--webserver_port=42485
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:39737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:17:55.627436 1231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:55.627791 1231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:55.627887 1231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:55.634971 1231 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 15:17:55.635066 1231 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 15:17:55.635103 1231 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 15:17:55.635149 1231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:17:55.635195 1231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:55.635263 1231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:55.635324 1231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:17:55.635377 1231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:17:55.645337 1231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:39737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=42485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:55.646872 1231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:55.648317 1231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:55.660452 1237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:55.660497 1241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:55.660539 1238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:55.661254 1231 server_base.cc:1034] running on GCE node
I20260809 15:17:55.662032 1231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:55.663337 1231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:55.664559 1231 hybrid_clock.cc:648] HybridClock initialized: now 1786288675664471 us; error 50 us; skew 500 ppm
I20260809 15:17:55.664827 1231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:55 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:55.668032 1231 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:17:55.669565 1231 webserver.cc:460] Webserver started at http://127.15.11.190:42485/ using document root <none> and password file <none>
I20260809 15:17:55.670195 1231 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:55.670285 1231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:55.675743 1231 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:17:55.678807 1250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:55.680121 1231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:17:55.680346 1231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "bdee2bb4a55442fea6968c6e47c46c70"
format_stamp: "Formatted at 2026-08-09 15:17:09 on dist-test-slave-8w9v"
server_key: "f6b9c22925f755a77a2e506e10858f31"
server_key_iv: "3425793b9cb49f0d34843bedd482ac26"
server_key_version: "encryptionkey@0"
I20260809 15:17:55.681326 1231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:55.693781 1231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:55.697613 1231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:55.697885 1231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:55.717341 1231 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:39737
I20260809 15:17:55.717387 1307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:39737 every 8 connection(s)
I20260809 15:17:55.718832 1231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:17:55.719502 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 1231
I20260809 15:17:55.720606 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:38481
--local_ip_for_outbound_sockets=127.15.11.129
--tserver_master_addrs=127.15.11.190:39737
--webserver_port=43875
--webserver_interface=127.15.11.129
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:17:55.726002 1308 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:17:55.729210 1308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Bootstrap starting.
I20260809 15:17:55.752383 1308 log.cc:824] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:55.768496 1308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:17:55.769042 1308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Bootstrap complete.
I20260809 15:17:55.777967 1308 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:55.778847 1308 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdee2bb4a55442fea6968c6e47c46c70, State: Initialized, Role: FOLLOWER
I20260809 15:17:55.779574 1308 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:55.779778 1308 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:17:55.779922 1308 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:17:55.780092 1308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:17:55.783905 1308 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:55.784498 1308 leader_election.cc:302] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdee2bb4a55442fea6968c6e47c46c70; no voters:
I20260809 15:17:55.785226 1308 leader_election.cc:288] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 15:17:55.785367 1312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:17:55.786693 1312 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [term 3 LEADER]: Becoming Leader. State: Replica: bdee2bb4a55442fea6968c6e47c46c70, State: Running, Role: LEADER
I20260809 15:17:55.787336 1312 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } }
I20260809 15:17:55.787571 1308 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:17:55.789981 1314 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdee2bb4a55442fea6968c6e47c46c70. Latest consensus state: current_term: 3 leader_uuid: "bdee2bb4a55442fea6968c6e47c46c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } } }
I20260809 15:17:55.790385 1314 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: This master's current role is: LEADER
I20260809 15:17:55.791006 1313 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bdee2bb4a55442fea6968c6e47c46c70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdee2bb4a55442fea6968c6e47c46c70" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 39737 } } }
I20260809 15:17:55.791340 1313 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70 [sys.catalog]: This master's current role is: LEADER
I20260809 15:17:55.799304 1319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:17:55.809552 1319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:17:55.811290 1319 catalog_manager.cc:1259] Loaded cluster ID: 4a4333384c404b418ab0c5e25a0730c7
I20260809 15:17:55.811455 1319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:17:55.816217 1319 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:17:55.818354 1319 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Loaded TSK: 0
I20260809 15:17:55.818612 1319 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bdee2bb4a55442fea6968c6e47c46c70: Loaded TSK: 1
F20260809 15:17:55.819964 1319 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
I20260809 15:17:55.821681 1330 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f5b230ea9ad google::LogMessage::Fail() at ??:0
@ 0x7f5b230ed88a google::LogMessage::SendToLog() at ??:0
@ 0x7f5b230ea4de google::LogMessage::Flush() at ??:0
@ 0x7f5b230eba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5b26e4e7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5b26e3d73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f5b38b191c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f5b38b18783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f5b38ba42a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f5b38ba40c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f5b38bee5e9 std::function<>::operator()() at ??:0
@ 0x7f5b38b1ccdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5b38b1ac36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5b38ba2610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f5b38ba2439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5b25d36345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5b25d3991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f5b25d39799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5b25d14eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5b28fee6db start_thread at ??:0
@ 0x7f5b21f8461f clone at ??:0
W20260809 15:17:56.127669 1310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:56.127939 1310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:56.128093 1310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:56.135241 1310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:56.135345 1310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:56.135524 1310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:17:56.145148 1310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:38481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=43875
--tserver_master_addrs=127.15.11.190:39737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:56.146394 1310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:56.147732 1310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:56.161258 1358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:56.161224 1359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:56.161304 1361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:56.162222 1310 server_base.cc:1034] running on GCE node
I20260809 15:17:56.162776 1310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:56.163682 1310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:56.164914 1310 hybrid_clock.cc:648] HybridClock initialized: now 1786288676164871 us; error 58 us; skew 500 ppm
I20260809 15:17:56.165241 1310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:56 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288676, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:56.168402 1310 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:17:56.169754 1310 webserver.cc:460] Webserver started at http://127.15.11.129:43875/ using document root <none> and password file <none>
I20260809 15:17:56.170459 1310 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:56.170609 1310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:56.175856 1310 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:17:56.178941 1369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:56.180274 1310 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:17:56.180455 1310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "a02e080e98014736aabafd5900943ac4"
format_stamp: "Formatted at 2026-08-09 15:17:10 on dist-test-slave-8w9v"
server_key: "855aa81242813d5bc63020cffbbc8771"
server_key_iv: "37d04c40f9dfc7ef924d483da1af4b66"
server_key_version: "encryptionkey@0"
I20260809 15:17:56.181406 1310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:56.202808 1310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:56.206688 1310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:56.206980 1310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:56.207991 1310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:17:56.209846 1377 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 15:17:56.215662 1310 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 15:17:56.215811 1310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 15:17:56.215968 1310 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 15:17:56.219163 1310 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 15:17:56.219267 1310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.000s
I20260809 15:17:56.219624 1377 tablet_bootstrap.cc:492] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Bootstrap starting.
I20260809 15:17:56.247931 1377 log.cc:824] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:17:56.262804 1310 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:38481
I20260809 15:17:56.262964 1490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:38481 every 8 connection(s)
I20260809 15:17:56.264219 1310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:17:56.264238 1377 tablet_bootstrap.cc:492] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:17:56.264806 1377 tablet_bootstrap.cc:492] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Bootstrap complete.
I20260809 15:17:56.266448 1377 ts_tablet_manager.cc:1397] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Time spent bootstrapping tablet: real 0.047s user 0.017s sys 0.027s
I20260809 15:17:56.269345 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:56.267257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:41779)
0809 15:17:56.268086 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:56.268447 (+ 361us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":380,"threads_started":1}
W20260809 15:17:56.270051 1491 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:39737 (0 consecutive failures): Network error: Failed to ping master at 127.15.11.190:39737: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
I20260809 15:17:56.270864 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:56.270554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:47277)
0809 15:17:56.270696 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:56.270764 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 15:17:56.271710 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:56.271341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:40933)
0809 15:17:56.271490 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:56.271576 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 15:17:56.271965 1491 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:17:56.272531 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 1310
I20260809 15:17:56.273277 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:36485
--local_ip_for_outbound_sockets=127.15.11.130
--tserver_master_addrs=127.15.11.190:39737
--webserver_port=33099
--webserver_interface=127.15.11.130
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:17:56.275461 1377 raft_consensus.cc:348] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:56.276911 1377 raft_consensus.cc:729] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02e080e98014736aabafd5900943ac4, State: Initialized, Role: FOLLOWER
I20260809 15:17:56.277657 1377 consensus_queue.cc:260] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:56.279160 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:56.278849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:52237)
0809 15:17:56.278999 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:56.279064 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 15:17:56.279379 1377 ts_tablet_manager.cc:1428] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260809 15:17:56.280277 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:56.279927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:45221)
0809 15:17:56.280057 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:56.280165 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 15:17:56.610929 1497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:56.611238 1497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:56.611331 1497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:56.618500 1497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:56.618621 1497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:56.618736 1497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:17:56.627969 1497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:36485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=33099
--tserver_master_addrs=127.15.11.190:39737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:56.629176 1497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:56.630434 1497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:56.644167 1503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:56.644268 1506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:56.644222 1504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:56.645156 1497 server_base.cc:1034] running on GCE node
I20260809 15:17:56.645715 1497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:56.646597 1497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:56.647812 1497 hybrid_clock.cc:648] HybridClock initialized: now 1786288676647765 us; error 33 us; skew 500 ppm
I20260809 15:17:56.648104 1497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:56 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288676, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:56.651194 1497 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:17:56.652649 1497 webserver.cc:460] Webserver started at http://127.15.11.130:33099/ using document root <none> and password file <none>
I20260809 15:17:56.653326 1497 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:56.653483 1497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:56.658787 1497 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:17:56.661473 1513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:56.662938 1497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:17:56.663128 1497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "b90b5679f1d74193bdcdfa82ddd24d91"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "f9ef2b9a8973ad2aa768fcc2740c224f"
server_key_iv: "4adbb7801be76edf48b4df50f80b163f"
server_key_version: "encryptionkey@0"
I20260809 15:17:56.664362 1497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:56.697264 1497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:56.701254 1497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:56.701551 1497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:56.702577 1497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:17:56.704093 1497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:17:56.704241 1497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 15:17:56.704356 1497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:17:56.704442 1497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:56.738057 1497 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:36485
I20260809 15:17:56.738109 1633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:36485 every 8 connection(s)
I20260809 15:17:56.739514 1497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:17:56.744331 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:56.742462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:54419)
0809 15:17:56.743164 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:56.743410 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":264,"threads_started":1}
I20260809 15:17:56.744637 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 1497
W20260809 15:17:56.745090 1634 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:39737 (0 consecutive failures): Network error: Failed to ping master at 127.15.11.190:39737: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
I20260809 15:17:56.745429 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:44495
--local_ip_for_outbound_sockets=127.15.11.131
--tserver_master_addrs=127.15.11.190:39737
--webserver_port=32921
--webserver_interface=127.15.11.131
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:17:56.746147 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:56.745646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:40063)
0809 15:17:56.745880 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:56.745980 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260809 15:17:56.746978 1636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:56.746653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:55137)
0809 15:17:56.746790 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:56.746866 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 15:17:56.747308 1634 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:17:57.078341 1638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:17:57.078613 1638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:17:57.078791 1638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:17:57.086117 1638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:17:57.086244 1638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:17:57.086436 1638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:17:57.095860 1638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:44495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=32921
--tserver_master_addrs=127.15.11.190:39737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:17:57.097153 1638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:17:57.098496 1638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:17:57.111871 1644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:57.111822 1646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:17:57.111927 1643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:17:57.112746 1638 server_base.cc:1034] running on GCE node
I20260809 15:17:57.113348 1638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:17:57.114279 1638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:17:57.115486 1638 hybrid_clock.cc:648] HybridClock initialized: now 1786288677115422 us; error 54 us; skew 500 ppm
I20260809 15:17:57.115803 1638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:17:57 dist-test-slave-8w9v krb5kdc[32513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288677, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:17:57.119057 1638 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:17:57.120515 1638 webserver.cc:460] Webserver started at http://127.15.11.131:32921/ using document root <none> and password file <none>
I20260809 15:17:57.121148 1638 fs_manager.cc:359] Metadata directory not provided
I20260809 15:17:57.121249 1638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:17:57.126322 1638 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 15:17:57.128856 1653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:57.130139 1638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:17:57.130317 1638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "e91a48dd94454d64bafbe6831254927c"
format_stamp: "Formatted at 2026-08-09 15:17:11 on dist-test-slave-8w9v"
server_key: "62aba2f6297aa6ae24a8e31266fce711"
server_key_iv: "b7221bb679c2c627aa6ae130f306c045"
server_key_version: "encryptionkey@0"
I20260809 15:17:57.131263 1638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:17:57.146976 1638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:17:57.151153 1638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:17:57.151417 1638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:17:57.152395 1638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:17:57.153633 1638 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:17:57.153751 1638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:57.153887 1638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:17:57.153980 1638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:17:57.187888 1638 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:44495
I20260809 15:17:57.187955 1774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:44495 every 8 connection(s)
I20260809 15:17:57.189347 1638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:17:57.193845 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 1638
I20260809 15:17:57.194178 1777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:57.192212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:54325)
0809 15:17:57.192992 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:57.193236 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":274,"threads_started":1}
W20260809 15:17:57.194931 1775 heartbeater.cc:644] Failed to heartbeat to 127.15.11.190:39737 (0 consecutive failures): Network error: Failed to ping master at 127.15.11.190:39737: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
I20260809 15:17:57.195739 1777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:57.195419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:45187)
0809 15:17:57.195567 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:57.195644 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 15:17:57.196661 1777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:57.196352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:33277)
0809 15:17:57.196468 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:57.196540 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 15:17:57.196941 1775 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:17:57.281872 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:57.281150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:50443)
0809 15:17:57.281568 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:57.281677 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":209,"threads_started":1}
I20260809 15:17:57.677147 1781 raft_consensus.cc:482] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:17:57.677528 1781 raft_consensus.cc:504] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:57.679173 1781 leader_election.cc:288] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485), e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495)
Aug 09 15:17:57 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288676, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
Aug 09 15:17:57 dist-test-slave-8w9v krb5kdc[32513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288676, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:17:57.701792 1783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:57.679823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:57101 (local address 127.15.11.130:36485)
0809 15:17:57.681224 (+ 1401us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:57.681241 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:57.681288 (+ 47us) server_negotiation.cc:407] Connection header received
0809 15:17:57.683002 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:57.683012 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:57.683378 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:57.683718 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:57.685663 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:57.686967 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:57.688669 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:57.689297 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:57.689519 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:57.693609 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:57.693616 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:57.693628 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:57.693664 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:57.696744 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:57.698191 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:57.698201 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:57.698209 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:57.698470 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:57.698996 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:57.699001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:57.699005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:57.699697 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:57.699908 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:57.700971 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0809 15:17:57.701380 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1018,"thread_start_us":981,"threads_started":1}
I20260809 15:17:57.701792 1784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:57.680578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:59031 (local address 127.15.11.131:44495)
0809 15:17:57.682030 (+ 1452us) server_negotiation.cc:206] Beginning negotiation
0809 15:17:57.682040 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:17:57.682078 (+ 38us) server_negotiation.cc:407] Connection header received
0809 15:17:57.683514 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:17:57.683523 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:17:57.683848 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:17:57.684192 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:17:57.685667 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:57.686943 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:57.688683 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:17:57.689305 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:17:57.689536 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:57.693806 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:17:57.693812 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:17:57.693823 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:17:57.693856 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:17:57.696840 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:57.698191 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:57.698201 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:57.698209 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:57.698470 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:17:57.698996 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:17:57.699001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:17:57.699005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:17:57.699692 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:17:57.699880 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 15:17:57.700971 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0809 15:17:57.701405 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1287,"thread_start_us":1146,"threads_started":1}
I20260809 15:17:57.701786 1780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:57.679454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:36485 (local address 127.15.11.129:57101)
0809 15:17:57.679642 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:57.679682 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:57.680527 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:57.683833 (+ 3306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:57.683848 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:57.684705 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:17:57.685462 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:57.685483 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:57.687137 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:57.687149 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:57.688532 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:57.688544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:57.689525 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:57.689534 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:57.689791 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:57.690624 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:17:57.690641 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:17:57.693295 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:17:57.696927 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:57.696936 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:57.696951 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:57.698017 (+ 1066us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:57.698597 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:57.698602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:57.698605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:57.698868 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:57.699852 (+ 984us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:17:57.699861 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:17:57.700366 (+ 505us) client_negotiation.cc:769] Sending connection context
0809 15:17:57.700696 (+ 330us) client_negotiation.cc:240] Negotiation successful
0809 15:17:57.701411 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":293,"spinlock_wait_cycles":8192}
I20260809 15:17:57.701807 1782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:57.679823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:44495 (local address 127.15.11.129:59031)
0809 15:17:57.681715 (+ 1892us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:57.681732 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:17:57.681841 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:17:57.684113 (+ 2272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:17:57.684143 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:17:57.684849 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:17:57.685462 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:57.685482 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:57.687142 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:57.687146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:57.688540 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:17:57.688552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:17:57.689581 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:17:57.689589 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:17:57.689793 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:17:57.690562 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:17:57.690587 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:17:57.693546 (+ 2959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:17:57.697093 (+ 3547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:57.697102 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:57.697107 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:57.698017 (+ 910us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:57.698597 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:17:57.698602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:17:57.698605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:17:57.698872 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:17:57.699852 (+ 980us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:17:57.699861 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:17:57.700366 (+ 505us) client_negotiation.cc:769] Sending connection context
0809 15:17:57.700697 (+ 331us) client_negotiation.cc:240] Negotiation successful
0809 15:17:57.701284 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1773,"mutex_wait_us":258,"spinlock_wait_cycles":52096,"thread_start_us":1605,"threads_started":1}
W20260809 15:17:57.705183 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
W20260809 15:17:57.705544 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
I20260809 15:17:57.705711 1374 leader_election.cc:302] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a02e080e98014736aabafd5900943ac4; no voters: b90b5679f1d74193bdcdfa82ddd24d91, e91a48dd94454d64bafbe6831254927c
I20260809 15:17:57.706140 1781 raft_consensus.cc:2726] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:17:57.749260 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:57.747891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:38271)
0809 15:17:57.748673 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:57.748913 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":520,"threads_started":1}
I20260809 15:17:58.198365 1786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:58.197556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:56563)
0809 15:17:58.198017 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:58.198138 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":212,"threads_started":1}
I20260809 15:17:58.283380 1788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:58.282770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:50163)
0809 15:17:58.283156 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:58.283231 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":200,"threads_started":1}
I20260809 15:17:58.750800 1789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:58.750158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:44163)
0809 15:17:58.750575 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:58.750652 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":226,"threads_started":1}
I20260809 15:17:59.199939 1790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:59.199294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:37765)
0809 15:17:59.199702 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:59.199777 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":218,"threads_started":1}
I20260809 15:17:59.285315 1791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:59.284500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:42949)
0809 15:17:59.285060 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:59.285137 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":236,"threads_started":1}
I20260809 15:17:59.347874 1792 raft_consensus.cc:482] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:17:59.348070 1792 raft_consensus.cc:504] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:17:59.348798 1792 leader_election.cc:288] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485), e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495)
W20260809 15:17:59.349751 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
W20260809 15:17:59.349972 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
I20260809 15:17:59.350047 1374 leader_election.cc:302] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a02e080e98014736aabafd5900943ac4; no voters: b90b5679f1d74193bdcdfa82ddd24d91, e91a48dd94454d64bafbe6831254927c
I20260809 15:17:59.350308 1792 raft_consensus.cc:2726] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:17:59.752573 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:17:59.751653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:52737)
0809 15:17:59.752211 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 15:17:59.752325 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":270,"threads_started":1}
I20260809 15:18:00.201387 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:00.200724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:53137)
0809 15:18:00.201161 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:00.201235 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":223,"threads_started":1}
I20260809 15:18:00.286988 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:00.286255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:59091)
0809 15:18:00.286726 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:00.286807 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":265,"threads_started":1}
I20260809 15:18:00.754199 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:00.753458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:37981)
0809 15:18:00.753932 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:00.754019 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":236,"threads_started":1}
I20260809 15:18:01.203025 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:01.202238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:54805)
0809 15:18:01.202751 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:01.202830 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":280,"threads_started":1}
I20260809 15:18:01.288779 1798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:01.288014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:47827)
0809 15:18:01.288506 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:01.288592 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":261,"threads_started":1}
I20260809 15:18:01.531703 1799 raft_consensus.cc:482] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:18:01.531935 1799 raft_consensus.cc:504] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:18:01.532704 1799 leader_election.cc:288] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485), e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495)
W20260809 15:18:01.533576 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
W20260809 15:18:01.533747 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
I20260809 15:18:01.533833 1374 leader_election.cc:302] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a02e080e98014736aabafd5900943ac4; no voters: b90b5679f1d74193bdcdfa82ddd24d91, e91a48dd94454d64bafbe6831254927c
I20260809 15:18:01.534054 1799 raft_consensus.cc:2726] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:18:01.756033 1800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:01.755273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:44063)
0809 15:18:01.755735 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:01.755821 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":245,"threads_started":1}
I20260809 15:18:02.204763 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:02.203947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:58043)
0809 15:18:02.204532 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:02.204611 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":357,"threads_started":1}
I20260809 15:18:02.290648 1802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:02.289715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:39789)
0809 15:18:02.290268 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:02.290408 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":293,"threads_started":1}
I20260809 15:18:02.757696 1803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:02.756955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:34323)
0809 15:18:02.757435 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:02.757522 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":236,"threads_started":1}
I20260809 15:18:03.206302 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:03.205622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:57965)
0809 15:18:03.206050 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:03.206128 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":225,"threads_started":1}
I20260809 15:18:03.292068 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:03.291405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:41881)
0809 15:18:03.291801 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:03.291875 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":217,"threads_started":1}
I20260809 15:18:03.759341 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:03.758570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:38643)
0809 15:18:03.759066 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:03.759148 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":223,"threads_started":1}
I20260809 15:18:04.208043 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:04.207272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:50993)
0809 15:18:04.207772 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:04.207871 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":250,"threads_started":1}
I20260809 15:18:04.293735 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:04.292979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:40789)
0809 15:18:04.293482 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:04.293574 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":282,"threads_started":1}
I20260809 15:18:04.760943 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:04.760294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:46233)
0809 15:18:04.760686 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:04.760764 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":197,"threads_started":1}
I20260809 15:18:04.773530 1811 raft_consensus.cc:482] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:18:04.773775 1811 raft_consensus.cc:504] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:18:04.774427 1811 leader_election.cc:288] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485), e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495)
W20260809 15:18:04.775336 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
W20260809 15:18:04.775568 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
I20260809 15:18:04.775643 1374 leader_election.cc:302] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a02e080e98014736aabafd5900943ac4; no voters: b90b5679f1d74193bdcdfa82ddd24d91, e91a48dd94454d64bafbe6831254927c
I20260809 15:18:04.775864 1811 raft_consensus.cc:2726] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:18:05.209638 1812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:05.208917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:38617)
0809 15:18:05.209361 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:05.209450 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":240,"threads_started":1}
I20260809 15:18:05.295380 1813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:05.294596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:55977)
0809 15:18:05.295080 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:05.295189 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":220,"threads_started":1}
I20260809 15:18:05.764747 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:05.761770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:40281)
0809 15:18:05.764508 (+ 2738us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:05.764587 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2629,"thread_start_us":2496,"threads_started":1}
I20260809 15:18:06.211211 1815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:06.210523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:40051)
0809 15:18:06.210988 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:06.211064 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":228,"threads_started":1}
I20260809 15:18:06.297187 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:06.296452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:43403)
0809 15:18:06.296907 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:06.296989 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":222,"threads_started":1}
I20260809 15:18:06.772758 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:06.765768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:54195)
0809 15:18:06.772493 (+ 6725us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:06.772575 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6610,"thread_start_us":6523,"threads_started":1}
I20260809 15:18:07.212903 1819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:07.212215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:34027)
0809 15:18:07.212644 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:07.212723 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":233,"threads_started":1}
I20260809 15:18:07.299022 1820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:07.298115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:52277)
0809 15:18:07.298753 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:07.298837 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":414,"threads_started":1}
I20260809 15:18:07.774364 1821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:07.773646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:52749)
0809 15:18:07.774100 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:07.774193 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":227,"threads_started":1}
I20260809 15:18:08.214587 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:08.213800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:46257)
0809 15:18:08.214326 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:08.214422 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":268,"threads_started":1}
I20260809 15:18:08.300581 1823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:08.299821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:45159)
0809 15:18:08.300324 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:08.300409 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":245,"threads_started":1}
I20260809 15:18:08.775972 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:08.775225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:53333)
0809 15:18:08.775708 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:08.775786 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":275,"threads_started":1}
I20260809 15:18:09.216461 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:09.215598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:41777)
0809 15:18:09.216098 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:09.216271 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":240,"threads_started":1}
I20260809 15:18:09.292443 1826 raft_consensus.cc:482] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:18:09.292639 1826 raft_consensus.cc:504] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b90b5679f1d74193bdcdfa82ddd24d91" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 36485 } } peers { permanent_uuid: "e91a48dd94454d64bafbe6831254927c" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 44495 } } peers { permanent_uuid: "a02e080e98014736aabafd5900943ac4" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 38481 } }
I20260809 15:18:09.293306 1826 leader_election.cc:288] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485), e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495)
W20260809 15:18:09.294425 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b90b5679f1d74193bdcdfa82ddd24d91 (127.15.11.130:36485): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
W20260809 15:18:09.294790 1374 leader_election.cc:341] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e91a48dd94454d64bafbe6831254927c (127.15.11.131:44495): Not found: Tablet not found: 1dca3295f096472aad5bed1c145ac01e
I20260809 15:18:09.294943 1374 leader_election.cc:302] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a02e080e98014736aabafd5900943ac4; no voters: b90b5679f1d74193bdcdfa82ddd24d91, e91a48dd94454d64bafbe6831254927c
I20260809 15:18:09.295213 1826 raft_consensus.cc:2726] T 1dca3295f096472aad5bed1c145ac01e P a02e080e98014736aabafd5900943ac4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:18:09.302100 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:09.301374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:60459)
0809 15:18:09.301789 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:09.301922 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":204,"threads_started":1}
I20260809 15:18:09.777946 1829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:09.777018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:46265)
0809 15:18:09.777652 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:09.777763 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":336,"threads_started":1}
I20260809 15:18:10.218302 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:10.217453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:42319)
0809 15:18:10.218025 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:10.218142 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":300,"threads_started":1}
I20260809 15:18:10.303678 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:10.302959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:56295)
0809 15:18:10.303394 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:10.303476 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":230,"threads_started":1}
I20260809 15:18:10.779506 1833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:10.778774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:47665)
0809 15:18:10.779223 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:10.779305 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":238,"threads_started":1}
I20260809 15:18:11.219965 1834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:11.219193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:60659)
0809 15:18:11.219669 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:11.219752 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":244,"threads_started":1}
I20260809 15:18:11.305575 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:11.304900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.129:50421)
0809 15:18:11.305310 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:11.305411 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":207,"threads_started":1}
I20260809 15:18:11.781258 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:11.780591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.130:58367)
0809 15:18:11.781005 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:11.781094 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":207,"threads_started":1}
I20260809 15:18:12.197890 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 1310
I20260809 15:18:12.219354 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:12.198897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.15.11.129:38481)
0809 15:18:12.199499 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:12.199508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:12.199542 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:18:12.199641 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:12.199649 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:12.199959 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:12.200158 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:12.201771 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:12.202850 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:12.204112 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:12.204724 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:12.204826 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:12.213493 (+ 8667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:12.213506 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:12.213522 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:12.213558 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:12.216726 (+ 3168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:12.217387 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:12.217397 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:12.217399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:12.217543 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:12.217882 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:12.217885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:12.217887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:12.218242 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:12.218481 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:12.218769 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 15:18:12.218982 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":310,"threads_started":1}
I20260809 15:18:12.220753 1486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:12.222434 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:12.221045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:39737 (local address 127.15.11.131:53687)
0809 15:18:12.221963 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:12.222169 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.190:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":612,"threads_started":1}
I20260809 15:18:12.315045 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 1310
I20260809 15:18:12.338513 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 1497
I20260809 15:18:12.351502 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:12.339380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54950 (local address 127.15.11.130:36485)
0809 15:18:12.339902 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:12.339907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:12.339924 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:18:12.339980 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:12.339983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:12.340044 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:12.340261 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:12.341534 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:12.342317 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:12.343479 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:12.343898 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:12.343991 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:12.346699 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:12.346704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:12.346707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:12.346736 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:12.349216 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:12.349788 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:12.349792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:12.349794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:12.349911 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:12.350236 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:12.350239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:12.350241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:12.350485 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:12.350678 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:12.350930 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 15:18:12.351084 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":271,"threads_started":1}
I20260809 15:18:12.352267 1629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:12.437529 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 1497
I20260809 15:18:12.460059 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 1638
I20260809 15:18:12.473436 1843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:12.461046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.15.11.131:44495)
0809 15:18:12.461627 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:12.461631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:12.461649 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:18:12.461703 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:12.461707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:12.461770 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:12.461859 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:12.463468 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:12.464372 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:12.465569 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:12.465965 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:12.466054 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:12.468585 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:12.468590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:12.468593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:12.468621 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:12.471123 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:12.471745 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:12.471749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:12.471751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:12.471899 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:12.472319 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:12.472322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:12.472324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:12.472568 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:12.472766 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:12.472991 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 15:18:12.473149 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":330,"threads_started":1}
I20260809 15:18:12.474193 1770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:12.558104 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 1638
2026-08-09T15:18:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64386 ms)
[----------] 30 tests from SecurityITest (300356 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:12 dist-test-slave-8w9v krb5kdc[1847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:12 dist-test-slave-8w9v krb5kdc[1847](info): set up 2 sockets
Aug 09 15:18:12 dist-test-slave-8w9v krb5kdc[1847](info): commencing operation
krb5kdc: starting...
W20260809 15:18:13.719789 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:13 dist-test-slave-8w9v krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:18:14.006294 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:42777
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:42777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:14.347587 1861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:14.347872 1861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:14.347957 1861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:14.354912 1861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:14.355005 1861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:14.355060 1861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:14.355144 1861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:14.355226 1861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:14.364836 1861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:42777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:42777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:14.366122 1861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:14.367512 1861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:14.379839 1869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:14.379940 1867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:14.379968 1866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:14.380429 1861 server_base.cc:1034] running on GCE node
I20260809 15:18:14.381428 1861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:14.382762 1861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:14.383987 1861 hybrid_clock.cc:648] HybridClock initialized: now 1786288694383936 us; error 47 us; skew 500 ppm
I20260809 15:18:14.384387 1861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:14 dist-test-slave-8w9v krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288694, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:14.387625 1861 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:18:14.389103 1861 webserver.cc:460] Webserver started at http://127.15.11.190:35065/ using document root <none> and password file <none>
I20260809 15:18:14.389735 1861 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:14.389835 1861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:14.390102 1861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:14.393013 1861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "cb58f178f50c4cd7ae7e7516b19e56ac"
format_stamp: "Formatted at 2026-08-09 15:18:14 on dist-test-slave-8w9v"
server_key: "790abb9bb7d21aa483f0016912d7ad2f"
server_key_iv: "1f58251d37f5a691961361905dbdf9a9"
server_key_version: "encryptionkey@0"
I20260809 15:18:14.393673 1861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "cb58f178f50c4cd7ae7e7516b19e56ac"
format_stamp: "Formatted at 2026-08-09 15:18:14 on dist-test-slave-8w9v"
server_key: "790abb9bb7d21aa483f0016912d7ad2f"
server_key_iv: "1f58251d37f5a691961361905dbdf9a9"
server_key_version: "encryptionkey@0"
I20260809 15:18:14.398840 1861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 15:18:14.402163 1876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:14.403437 1861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:18:14.403585 1861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "cb58f178f50c4cd7ae7e7516b19e56ac"
format_stamp: "Formatted at 2026-08-09 15:18:14 on dist-test-slave-8w9v"
server_key: "790abb9bb7d21aa483f0016912d7ad2f"
server_key_iv: "1f58251d37f5a691961361905dbdf9a9"
server_key_version: "encryptionkey@0"
I20260809 15:18:14.403759 1861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:14.424338 1861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:14.428260 1861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:14.428506 1861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:14.447122 1861 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:42777
I20260809 15:18:14.447145 1932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:42777 every 8 connection(s)
I20260809 15:18:14.448589 1861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:14.452425 1933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:14.455752 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 1861
I20260809 15:18:14.455976 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:14.456880 15406 external_mini_cluster.cc:1428] Setting key 532091b19df8308ea9da2b4338fd8705
I20260809 15:18:14.465649 1933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac: Bootstrap starting.
I20260809 15:18:14.469166 1933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:14.470155 1933 log.cc:824] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:18:14 dist-test-slave-8w9v krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:14.473215 1933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac: No bootstrap required, opened a new log
I20260809 15:18:14.479858 1936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:14.458973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51712 (local address 127.15.11.190:42777)
0809 15:18:14.459741 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:14.459752 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:14.459791 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:18:14.461464 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:14.461504 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:14.461988 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:14.462415 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:14.463687 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:14.465648 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:14.467051 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:14.468058 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:14.468305 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:14.472532 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:14.472539 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:14.472551 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:14.472584 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:14.475722 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:14.476672 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:14.476678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:14.476683 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:14.476845 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:14.477281 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:14.477284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:14.477286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:14.477804 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:14.478031 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:14.478490 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 15:18:14.478827 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":376,"threads_started":1}
I20260809 15:18:14.482355 1933 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb58f178f50c4cd7ae7e7516b19e56ac" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42777 } }
I20260809 15:18:14.482792 1933 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:14.482911 1933 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb58f178f50c4cd7ae7e7516b19e56ac, State: Initialized, Role: FOLLOWER
I20260809 15:18:14.483485 1933 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb58f178f50c4cd7ae7e7516b19e56ac" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42777 } }
I20260809 15:18:14.483716 1933 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:14.483841 1933 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:14.484015 1933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:14.486106 1933 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb58f178f50c4cd7ae7e7516b19e56ac" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42777 } }
I20260809 15:18:14.486595 1933 leader_election.cc:302] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb58f178f50c4cd7ae7e7516b19e56ac; no voters:
I20260809 15:18:14.487166 1933 leader_election.cc:288] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:14.487304 1938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:14.487991 1938 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [term 1 LEADER]: Becoming Leader. State: Replica: cb58f178f50c4cd7ae7e7516b19e56ac, State: Running, Role: LEADER
I20260809 15:18:14.488641 1938 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb58f178f50c4cd7ae7e7516b19e56ac" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42777 } }
I20260809 15:18:14.489239 1933 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:14.491642 1940 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb58f178f50c4cd7ae7e7516b19e56ac. Latest consensus state: current_term: 1 leader_uuid: "cb58f178f50c4cd7ae7e7516b19e56ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb58f178f50c4cd7ae7e7516b19e56ac" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42777 } } }
I20260809 15:18:14.492177 1940 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:14.491770 1939 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb58f178f50c4cd7ae7e7516b19e56ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb58f178f50c4cd7ae7e7516b19e56ac" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42777 } } }
I20260809 15:18:14.492532 1939 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:14.493973 1947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:14.499424 1947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:14.508261 1947 catalog_manager.cc:1347] Generated new cluster ID: 5235e674085b48f680443bfe5f104c85
I20260809 15:18:14.508368 1947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:14.533077 1947 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:14.534336 1947 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:14.541196 1947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac: Generated new TSK 0
I20260809 15:18:14.542023 1947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:18:14.619537 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42777
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:14.956692 1960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:14.956940 1960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:14.957032 1960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:14.963819 1960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:14.963923 1960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:14.964025 1960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:18:14.973140 1960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:14.974242 1960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:14.975580 1960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:14.988371 1965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:14.988423 1968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:14.988461 1966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:14.989579 1960 server_base.cc:1034] running on GCE node
I20260809 15:18:14.990227 1960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:14.991091 1960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:14.992313 1960 hybrid_clock.cc:648] HybridClock initialized: now 1786288694992263 us; error 36 us; skew 500 ppm
I20260809 15:18:14.992631 1960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:14 dist-test-slave-8w9v krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288694, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:14.995720 1960 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:18:14.997157 1960 webserver.cc:460] Webserver started at http://127.15.11.129:41567/ using document root <none> and password file <none>
I20260809 15:18:14.997848 1960 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:14.998014 1960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:14.998332 1960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:15.001294 1960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "4e8ee879333e4e75a51edc6128fcc7a1"
format_stamp: "Formatted at 2026-08-09 15:18:14 on dist-test-slave-8w9v"
server_key: "75f4530a4e6903a413566a07c77c2b28"
server_key_iv: "e348ffa33d3f2fb412918935cca68d71"
server_key_version: "encryptionkey@0"
I20260809 15:18:15.002126 1960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "4e8ee879333e4e75a51edc6128fcc7a1"
format_stamp: "Formatted at 2026-08-09 15:18:14 on dist-test-slave-8w9v"
server_key: "75f4530a4e6903a413566a07c77c2b28"
server_key_iv: "e348ffa33d3f2fb412918935cca68d71"
server_key_version: "encryptionkey@0"
I20260809 15:18:15.007169 1960 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 15:18:15.010655 1975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:15.011938 1960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:18:15.012183 1960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "4e8ee879333e4e75a51edc6128fcc7a1"
format_stamp: "Formatted at 2026-08-09 15:18:14 on dist-test-slave-8w9v"
server_key: "75f4530a4e6903a413566a07c77c2b28"
server_key_iv: "e348ffa33d3f2fb412918935cca68d71"
server_key_version: "encryptionkey@0"
I20260809 15:18:15.012404 1960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:15.025786 1960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:15.029418 1960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:15.029706 1960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:15.030663 1960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:15.031885 1960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:15.031987 1960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:15.032188 1960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:15.032284 1960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 15:18:15.065146 1960 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:40779
I20260809 15:18:15.065191 2096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:40779 every 8 connection(s)
I20260809 15:18:15.066465 1960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:18:15.073295 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 1960
I20260809 15:18:15.073527 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:18:15.074324 15406 external_mini_cluster.cc:1428] Setting key 5fde79206443298e397c402ded560102
Aug 09 15:18:15 dist-test-slave-8w9v krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288694, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:15.087227 2099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:15.069297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:46171 (local address 127.15.11.190:42777)
0809 15:18:15.069741 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:15.069745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:15.070597 (+ 852us) server_negotiation.cc:407] Connection header received
0809 15:18:15.072325 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:15.072329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:15.072403 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:15.072498 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:15.074052 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:15.075130 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:15.076968 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:15.077335 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:15.077442 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:15.081576 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:15.081580 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:15.081584 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:15.081614 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:15.084243 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:15.084952 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:15.084964 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:15.084966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:15.085090 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:15.085541 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:15.085545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:15.085547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:15.085792 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:15.085961 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:15.086615 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 15:18:15.086837 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":223,"threads_started":1}
I20260809 15:18:15.088205 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:15.069729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42777 (local address 127.15.11.129:46171)
0809 15:18:15.070439 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:15.070481 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:15.072023 (+ 1542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:15.072690 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:15.072697 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:15.073196 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:15.073871 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:15.073890 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:15.075309 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:15.075314 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:15.076816 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:15.076830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:15.077557 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:15.077567 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:15.077906 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:15.078640 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:18:15.078674 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:15.081333 (+ 2659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:15.084387 (+ 3054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:15.084403 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:15.084418 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:15.084816 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:15.085205 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:15.085210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:15.085213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:15.085415 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:15.085925 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:15.085932 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:15.086255 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 15:18:15.086509 (+ 254us) client_negotiation.cc:240] Negotiation successful
0809 15:18:15.087020 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":264,"threads_started":1}
I20260809 15:18:15.089933 2097 heartbeater.cc:347] Connected to a master server at 127.15.11.190:42777
I20260809 15:18:15.090255 2097 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:15.090973 2097 heartbeater.cc:511] Master 127.15.11.190:42777 requested a full tablet report, sending...
I20260809 15:18:15.093259 1895 ts_manager.cc:194] Registered new tserver with Master: 4e8ee879333e4e75a51edc6128fcc7a1 (127.15.11.129:40779)
I20260809 15:18:15.094693 1895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:46171
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:18:15.139139 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42777
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:15.476356 2103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:15.476611 2103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:15.476706 2103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:15.483513 2103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:15.483675 2103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:15.483808 2103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:18:15.493466 2103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:15.494691 2103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:15.496064 2103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:15.509181 2108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:15.509227 2109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:15.509276 2111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:15.510015 2103 server_base.cc:1034] running on GCE node
I20260809 15:18:15.510605 2103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:15.511480 2103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:15.512696 2103 hybrid_clock.cc:648] HybridClock initialized: now 1786288695512659 us; error 48 us; skew 500 ppm
I20260809 15:18:15.513062 2103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:15 dist-test-slave-8w9v krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288695, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:15.516525 2103 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:18:15.518038 2103 webserver.cc:460] Webserver started at http://127.15.11.130:44475/ using document root <none> and password file <none>
I20260809 15:18:15.518661 2103 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:15.518787 2103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:15.519065 2103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:15.521932 2103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "7986775052be4284a5732f8177c488c0"
format_stamp: "Formatted at 2026-08-09 15:18:15 on dist-test-slave-8w9v"
server_key: "d4c58294b5f3cc1dd94755d99ff12716"
server_key_iv: "a73e1838d83b65fe7a7e3fd7d2c61c0d"
server_key_version: "encryptionkey@0"
I20260809 15:18:15.522665 2103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "7986775052be4284a5732f8177c488c0"
format_stamp: "Formatted at 2026-08-09 15:18:15 on dist-test-slave-8w9v"
server_key: "d4c58294b5f3cc1dd94755d99ff12716"
server_key_iv: "a73e1838d83b65fe7a7e3fd7d2c61c0d"
server_key_version: "encryptionkey@0"
I20260809 15:18:15.527699 2103 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:18:15.531028 2118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:15.532379 2103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:18:15.532513 2103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "7986775052be4284a5732f8177c488c0"
format_stamp: "Formatted at 2026-08-09 15:18:15 on dist-test-slave-8w9v"
server_key: "d4c58294b5f3cc1dd94755d99ff12716"
server_key_iv: "a73e1838d83b65fe7a7e3fd7d2c61c0d"
server_key_version: "encryptionkey@0"
I20260809 15:18:15.532666 2103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:15.545186 2103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:15.549501 2103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:15.549738 2103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:15.550608 2103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:15.551800 2103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:15.551869 2103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:15.551957 2103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:15.552003 2103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:15.584789 2103 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:42837
I20260809 15:18:15.584838 2238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:42837 every 8 connection(s)
I20260809 15:18:15.586082 2103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:18:15.595510 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 2103
I20260809 15:18:15.595713 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:18:15.596220 15406 external_mini_cluster.cc:1428] Setting key feefa8be9fd9e637f36d7ff3b5db0d3c
Aug 09 15:18:15 dist-test-slave-8w9v krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288695, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:15.607108 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:15.588464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:55403 (local address 127.15.11.190:42777)
0809 15:18:15.588904 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:15.588908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:15.589984 (+ 1076us) server_negotiation.cc:407] Connection header received
0809 15:18:15.591889 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:15.591895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:15.591993 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:15.592113 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:15.593634 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:15.594369 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:15.595973 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:15.596562 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:15.596696 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:15.600375 (+ 3679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:15.600384 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:15.600399 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:15.600447 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:15.603893 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:15.604630 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:15.604638 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:15.604643 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:15.604799 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:15.605211 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:15.605218 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:15.605224 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:15.605576 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:15.605720 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:15.606357 (+ 637us) server_negotiation.cc:299] Negotiation successful
0809 15:18:15.606629 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":240,"threads_started":1}
I20260809 15:18:15.607818 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:15.588994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42777 (local address 127.15.11.130:55403)
0809 15:18:15.589820 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:15.589877 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:15.591627 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:15.592366 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:15.592375 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:15.592853 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:15.593465 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:15.593481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:15.594530 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:15.594534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:15.595836 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:15.595849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:15.596794 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:15.596801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:15.597033 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:15.597723 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:18:15.597750 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:15.600091 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:15.604044 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:15.604051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:15.604062 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:15.604518 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:15.604906 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:15.604909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:15.604911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:15.605083 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:15.605688 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:15.605694 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:15.605992 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 15:18:15.606198 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 15:18:15.606595 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":284,"threads_started":1}
I20260809 15:18:15.609382 2239 heartbeater.cc:347] Connected to a master server at 127.15.11.190:42777
I20260809 15:18:15.609668 2239 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:15.610345 2239 heartbeater.cc:511] Master 127.15.11.190:42777 requested a full tablet report, sending...
I20260809 15:18:15.611858 1895 ts_manager.cc:194] Registered new tserver with Master: 7986775052be4284a5732f8177c488c0 (127.15.11.130:42837)
I20260809 15:18:15.612709 1895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:55403
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:18:15.661646 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42777
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:43723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:16.001871 2245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:16.002247 2245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:16.002437 2245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:16.010110 2245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:16.010296 2245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:16.010486 2245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:18:16.020681 2245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:43723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:42777
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:16.022092 2245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:16.023550 2245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:16.038091 2251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:16.038023 2254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:16.038064 2252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:16.039433 2245 server_base.cc:1034] running on GCE node
I20260809 15:18:16.040084 2245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:16.041127 2245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:16.042377 2245 hybrid_clock.cc:648] HybridClock initialized: now 1786288696042324 us; error 58 us; skew 500 ppm
I20260809 15:18:16.042718 2245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:16 dist-test-slave-8w9v krb5kdc[1847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288696, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:16.046305 2245 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:18:16.047884 2245 webserver.cc:460] Webserver started at http://127.15.11.131:43829/ using document root <none> and password file <none>
I20260809 15:18:16.048657 2245 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:16.048811 2245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:16.049142 2245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:16.052070 2245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "ac914dc990d14ddd9bf9c62aaeaa6272"
format_stamp: "Formatted at 2026-08-09 15:18:16 on dist-test-slave-8w9v"
server_key: "5da178008d85095915757028534cbf0b"
server_key_iv: "923fe9e3721eff5a092d1a6736cbafb8"
server_key_version: "encryptionkey@0"
I20260809 15:18:16.052918 2245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "ac914dc990d14ddd9bf9c62aaeaa6272"
format_stamp: "Formatted at 2026-08-09 15:18:16 on dist-test-slave-8w9v"
server_key: "5da178008d85095915757028534cbf0b"
server_key_iv: "923fe9e3721eff5a092d1a6736cbafb8"
server_key_version: "encryptionkey@0"
I20260809 15:18:16.058019 2245 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:18:16.061446 2261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:16.062757 2245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:18:16.062955 2245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "ac914dc990d14ddd9bf9c62aaeaa6272"
format_stamp: "Formatted at 2026-08-09 15:18:16 on dist-test-slave-8w9v"
server_key: "5da178008d85095915757028534cbf0b"
server_key_iv: "923fe9e3721eff5a092d1a6736cbafb8"
server_key_version: "encryptionkey@0"
I20260809 15:18:16.063228 2245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:16.080716 2245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:16.084578 2245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:16.084874 2245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:16.085794 2245 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:16.087127 2245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:16.087232 2245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:16.087394 2245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:16.087467 2245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:16.097501 2097 heartbeater.cc:503] Master 127.15.11.190:42777 was elected leader, sending a full tablet report...
I20260809 15:18:16.123672 2245 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:42547
I20260809 15:18:16.123756 2379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:42547 every 8 connection(s)
I20260809 15:18:16.125046 2245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:18:16.129657 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 2245
I20260809 15:18:16.129866 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:18:16.130479 15406 external_mini_cluster.cc:1428] Setting key 778b522aa7af23733f5f5a0279669521
Aug 09 15:18:16 dist-test-slave-8w9v krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288696, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:16.145133 2382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.127475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:47341 (local address 127.15.11.190:42777)
0809 15:18:16.127975 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.127982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.128973 (+ 991us) server_negotiation.cc:407] Connection header received
0809 15:18:16.131388 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.131395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.131496 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.131608 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:16.133532 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.134353 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.135823 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.136323 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.136418 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.139674 (+ 3256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:16.139679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:16.139682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:16.139710 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:16.142223 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.142828 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.142832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.142834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.142958 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.143313 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.143316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.143317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.143574 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:16.143692 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.144488 (+ 796us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.144691 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":218,"threads_started":1}
I20260809 15:18:16.146262 2383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.128030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42777 (local address 127.15.11.131:47341)
0809 15:18:16.128809 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:16.128852 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:16.131137 (+ 2285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:16.131867 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:16.131875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:16.132590 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:16.133327 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.133352 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.134506 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.134510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.135699 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.135710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.136528 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.136532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.136720 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.137408 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:18:16.137430 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:16.139503 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:16.142339 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:16.142345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:16.142355 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:16.142734 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:16.143045 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:16.143048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:16.143050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:16.143218 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:16.143678 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:16.143683 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:16.144020 (+ 337us) client_negotiation.cc:769] Sending connection context
0809 15:18:16.144372 (+ 352us) client_negotiation.cc:240] Negotiation successful
0809 15:18:16.144853 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":276,"threads_started":1}
I20260809 15:18:16.147460 2380 heartbeater.cc:347] Connected to a master server at 127.15.11.190:42777
I20260809 15:18:16.147716 2380 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:16.148497 2380 heartbeater.cc:511] Master 127.15.11.190:42777 requested a full tablet report, sending...
I20260809 15:18:16.149832 1895 ts_manager.cc:194] Registered new tserver with Master: ac914dc990d14ddd9bf9c62aaeaa6272 (127.15.11.131:42547)
I20260809 15:18:16.150656 1895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:47341
I20260809 15:18:16.158179 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:18:16.174805 2382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.162697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.15.11.190:42777)
0809 15:18:16.162940 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.162949 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.163362 (+ 413us) server_negotiation.cc:407] Connection header received
0809 15:18:16.163502 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.163506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.163587 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.163666 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:16.165256 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.165977 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.167354 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.167798 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.167915 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.170108 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:16.170112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:16.170115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:16.170144 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:16.172577 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.173166 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.173169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.173171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.173285 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.173571 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.173573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.173575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.173847 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:16.174024 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.174293 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.174453 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:18:16.178607 1895 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:18:16.185526 1895 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:18:16.221531 2393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.197294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:40779 (local address 127.0.0.1:54876)
0809 15:18:16.204324 (+ 7030us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:16.204388 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:16.204619 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:16.213000 (+ 8381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:16.213007 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:16.213045 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:16.213499 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.213511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.216010 (+ 2499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.216017 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.218620 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.218635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.220430 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.220435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.220616 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.220683 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 15:18:16.220890 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 15:18:16.221084 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":301,"threads_started":1}
I20260809 15:18:16.221719 2396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.198196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:42837 (local address 127.0.0.1:34196)
0809 15:18:16.213049 (+ 14853us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:16.213069 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:16.213194 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:16.214162 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:16.214170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:16.214207 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:16.214743 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.214756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.216734 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.216739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.218458 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.218470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.220832 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.220842 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.221036 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.221066 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:18:16.221139 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 15:18:16.221261 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14726,"mutex_wait_us":3,"thread_start_us":16070,"threads_started":1}
I20260809 15:18:16.222290 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.198962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.15.11.131:42547)
0809 15:18:16.212560 (+ 13598us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.212567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.214764 (+ 2197us) server_negotiation.cc:407] Connection header received
0809 15:18:16.214921 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.214933 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.215078 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.215270 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:16.215897 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.217627 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.219597 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.220891 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.221014 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.221237 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.221901 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.222047 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13439,"thread_start_us":13274,"threads_started":1}
I20260809 15:18:16.222843 2397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.198378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.15.11.130:42837)
0809 15:18:16.211420 (+ 13042us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.211433 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.213337 (+ 1904us) server_negotiation.cc:407] Connection header received
0809 15:18:16.213455 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.213463 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.213695 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.213905 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:16.214945 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.216563 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.218625 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.220198 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.222044 (+ 1846us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.222242 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.222334 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.222506 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12778,"thread_start_us":13050,"threads_started":1}
I20260809 15:18:16.222860 2395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.198410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:42547 (local address 127.0.0.1:58238)
0809 15:18:16.214614 (+ 16204us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:16.214638 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:16.214766 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:16.215256 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:16.215261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:16.215286 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:16.215710 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.215729 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.217790 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.217795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.219449 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.219460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.221404 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.221409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.221526 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.221549 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:18:16.222505 (+ 956us) client_negotiation.cc:240] Negotiation successful
0809 15:18:16.222633 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16073,"thread_start_us":14334,"threads_started":1}
I20260809 15:18:16.222841 2394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.197698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.15.11.129:40779)
0809 15:18:16.212219 (+ 14521us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.212229 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.212266 (+ 37us) server_negotiation.cc:407] Connection header received
0809 15:18:16.212387 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.212398 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.212706 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.213705 (+ 999us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:16.214133 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.215857 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.218778 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.220198 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.222080 (+ 1882us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.222285 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.222346 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.222505 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14313,"thread_start_us":14160,"threads_started":1}
I20260809 15:18:16.226517 2171 tablet_service.cc:1467] Processing CreateTablet for tablet 20803b8831d943938af1c5eeb6095a67 (DEFAULT_TABLE table=test-table [id=40625f2c868245dea61799a418fd3b45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:16.226670 2027 tablet_service.cc:1467] Processing CreateTablet for tablet 20803b8831d943938af1c5eeb6095a67 (DEFAULT_TABLE table=test-table [id=40625f2c868245dea61799a418fd3b45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:16.226666 2310 tablet_service.cc:1467] Processing CreateTablet for tablet 20803b8831d943938af1c5eeb6095a67 (DEFAULT_TABLE table=test-table [id=40625f2c868245dea61799a418fd3b45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:16.227820 2171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20803b8831d943938af1c5eeb6095a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:16.228081 2310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20803b8831d943938af1c5eeb6095a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:16.228077 2027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20803b8831d943938af1c5eeb6095a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:16.240021 2399 tablet_bootstrap.cc:492] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: Bootstrap starting.
I20260809 15:18:16.244076 2399 tablet_bootstrap.cc:654] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:16.244261 2401 tablet_bootstrap.cc:492] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: Bootstrap starting.
I20260809 15:18:16.245669 2399 log.cc:824] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:16.247222 2400 tablet_bootstrap.cc:492] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: Bootstrap starting.
I20260809 15:18:16.248109 2401 tablet_bootstrap.cc:654] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:16.249339 2399 tablet_bootstrap.cc:492] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: No bootstrap required, opened a new log
I20260809 15:18:16.249511 2401 log.cc:824] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:16.249657 2399 ts_tablet_manager.cc:1397] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 15:18:16.250794 2400 tablet_bootstrap.cc:654] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:16.252092 2400 log.cc:824] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:16.252969 2401 tablet_bootstrap.cc:492] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: No bootstrap required, opened a new log
I20260809 15:18:16.253247 2401 ts_tablet_manager.cc:1397] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:18:16.255189 2400 tablet_bootstrap.cc:492] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: No bootstrap required, opened a new log
I20260809 15:18:16.255468 2400 ts_tablet_manager.cc:1397] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 15:18:16.258023 2399 raft_consensus.cc:348] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } }
I20260809 15:18:16.258416 2399 raft_consensus.cc:374] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:16.258524 2399 raft_consensus.cc:729] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7986775052be4284a5732f8177c488c0, State: Initialized, Role: FOLLOWER
I20260809 15:18:16.259193 2399 consensus_queue.cc:260] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } }
I20260809 15:18:16.264489 2401 raft_consensus.cc:348] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } }
I20260809 15:18:16.265007 2401 raft_consensus.cc:374] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:16.265033 2239 heartbeater.cc:503] Master 127.15.11.190:42777 was elected leader, sending a full tablet report...
I20260809 15:18:16.265131 2401 raft_consensus.cc:729] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac914dc990d14ddd9bf9c62aaeaa6272, State: Initialized, Role: FOLLOWER
I20260809 15:18:16.265831 2399 ts_tablet_manager.cc:1428] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
I20260809 15:18:16.265794 2401 consensus_queue.cc:260] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } }
I20260809 15:18:16.266942 2380 heartbeater.cc:503] Master 127.15.11.190:42777 was elected leader, sending a full tablet report...
I20260809 15:18:16.267566 2401 ts_tablet_manager.cc:1428] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 15:18:16.268186 2400 raft_consensus.cc:348] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } }
I20260809 15:18:16.268682 2400 raft_consensus.cc:374] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:16.268798 2400 raft_consensus.cc:729] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e8ee879333e4e75a51edc6128fcc7a1, State: Initialized, Role: FOLLOWER
I20260809 15:18:16.269450 2400 consensus_queue.cc:260] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } }
I20260809 15:18:16.273439 2400 ts_tablet_manager.cc:1428] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
W20260809 15:18:16.318900 2098 tablet.cc:2369] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:18:16.337980 2240 tablet.cc:2369] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:18:16.376747 2381 tablet.cc:2369] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:16.424407 2408 raft_consensus.cc:482] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:18:16.424811 2408 raft_consensus.cc:504] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } }
I20260809 15:18:16.427278 2408 leader_election.cc:288] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac914dc990d14ddd9bf9c62aaeaa6272 (127.15.11.131:42547), 7986775052be4284a5732f8177c488c0 (127.15.11.130:42837)
I20260809 15:18:16.440207 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.427669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:59825 (local address 127.15.11.131:42547)
0809 15:18:16.427922 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.427931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.433023 (+ 5092us) server_negotiation.cc:407] Connection header received
0809 15:18:16.433275 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.433285 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.433379 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.433483 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:16.434304 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.435099 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.437886 (+ 2787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.439115 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.439303 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.439385 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.439785 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.439930 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 15:18:16.440331 2397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.428300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:34857 (local address 127.15.11.130:42837)
0809 15:18:16.428578 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.428587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.433862 (+ 5275us) server_negotiation.cc:407] Connection header received
0809 15:18:16.434029 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.434034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.434107 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.434205 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:16.434749 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.435444 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.438068 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.439331 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.439490 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.439538 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.439908 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.440033 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 15:18:16.440615 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.427548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:42547 (local address 127.15.11.129:59825)
0809 15:18:16.432897 (+ 5349us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:16.432914 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:16.433109 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:16.433547 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:16.433553 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:16.433594 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:16.434101 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.434118 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.435247 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.435254 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.437705 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.437722 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.439341 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.439346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.439587 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.439630 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 15:18:16.439758 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 15:18:16.439908 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5212,"mutex_wait_us":4,"thread_start_us":5816,"threads_started":1}
I20260809 15:18:16.440665 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.427585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:42837 (local address 127.15.11.129:34857)
0809 15:18:16.433741 (+ 6156us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:16.433758 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:16.433909 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:16.434219 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:16.434224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:16.434246 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:16.434599 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.434607 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.435602 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.435607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.437903 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:16.437916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.439552 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.439559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:16.439750 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.439782 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:18:16.439870 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:18:16.439967 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6024,"mutex_wait_us":78,"thread_start_us":4956,"threads_started":1}
I20260809 15:18:16.441720 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20803b8831d943938af1c5eeb6095a67" candidate_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" is_pre_election: true
I20260809 15:18:16.441841 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20803b8831d943938af1c5eeb6095a67" candidate_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7986775052be4284a5732f8177c488c0" is_pre_election: true
I20260809 15:18:16.442288 2331 raft_consensus.cc:2454] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e8ee879333e4e75a51edc6128fcc7a1 in term 0.
I20260809 15:18:16.442461 2191 raft_consensus.cc:2454] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e8ee879333e4e75a51edc6128fcc7a1 in term 0.
I20260809 15:18:16.442957 1976 leader_election.cc:302] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e8ee879333e4e75a51edc6128fcc7a1, ac914dc990d14ddd9bf9c62aaeaa6272; no voters:
I20260809 15:18:16.443437 2408 raft_consensus.cc:2781] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:18:16.443612 2408 raft_consensus.cc:482] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:18:16.443725 2408 raft_consensus.cc:3037] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:16.446349 2408 raft_consensus.cc:504] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } }
I20260809 15:18:16.447182 2408 leader_election.cc:288] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [CANDIDATE]: Term 1 election: Requested vote from peers ac914dc990d14ddd9bf9c62aaeaa6272 (127.15.11.131:42547), 7986775052be4284a5732f8177c488c0 (127.15.11.130:42837)
I20260809 15:18:16.447733 2191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20803b8831d943938af1c5eeb6095a67" candidate_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7986775052be4284a5732f8177c488c0"
I20260809 15:18:16.447773 2331 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20803b8831d943938af1c5eeb6095a67" candidate_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272"
I20260809 15:18:16.447981 2191 raft_consensus.cc:3037] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:16.448050 2331 raft_consensus.cc:3037] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:16.450611 2331 raft_consensus.cc:2454] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e8ee879333e4e75a51edc6128fcc7a1 in term 1.
I20260809 15:18:16.451047 1976 leader_election.cc:302] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e8ee879333e4e75a51edc6128fcc7a1, ac914dc990d14ddd9bf9c62aaeaa6272; no voters:
I20260809 15:18:16.451360 2408 raft_consensus.cc:2781] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:16.451406 2191 raft_consensus.cc:2454] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e8ee879333e4e75a51edc6128fcc7a1 in term 1.
I20260809 15:18:16.452477 2408 raft_consensus.cc:686] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 1 LEADER]: Becoming Leader. State: Replica: 4e8ee879333e4e75a51edc6128fcc7a1, State: Running, Role: LEADER
I20260809 15:18:16.453154 2408 consensus_queue.cc:237] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } }
I20260809 15:18:16.458302 1895 catalog_manager.cc:5495] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e8ee879333e4e75a51edc6128fcc7a1 (127.15.11.129). New cstate: current_term: 1 leader_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e8ee879333e4e75a51edc6128fcc7a1" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 40779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 } health_report { overall_health: UNKNOWN } } }
I20260809 15:18:16.522600 2394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.517416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54878 (local address 127.15.11.129:40779)
0809 15:18:16.517614 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.517618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.517637 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:18:16.517729 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.517733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.517824 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.517959 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:18:16.518318 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.519076 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.520559 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.520968 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.521060 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.521569 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:18:16.522093 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:18:16.522206 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.522324 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.522403 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:18:16.531090 2331 raft_consensus.cc:1261] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [term 1 FOLLOWER]: Refusing update from remote peer 4e8ee879333e4e75a51edc6128fcc7a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:16.531095 2191 raft_consensus.cc:1261] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [term 1 FOLLOWER]: Refusing update from remote peer 4e8ee879333e4e75a51edc6128fcc7a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:16.532090 2411 consensus_queue.cc:1035] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac914dc990d14ddd9bf9c62aaeaa6272" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 42547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:16.532514 2408 consensus_queue.cc:1035] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7986775052be4284a5732f8177c488c0" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 42837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:16.564036 1894 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:16.564404 1894 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:16.568215 1894 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb58f178f50c4cd7ae7e7516b19e56ac: Sending DeleteTablet for 3 replicas of tablet 20803b8831d943938af1c5eeb6095a67
I20260809 15:18:16.569478 2171 tablet_service.cc:1514] Processing DeleteTablet for tablet 20803b8831d943938af1c5eeb6095a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:16 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:34196
I20260809 15:18:16.569478 2310 tablet_service.cc:1514] Processing DeleteTablet for tablet 20803b8831d943938af1c5eeb6095a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:16 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:58238
I20260809 15:18:16.570240 2027 tablet_service.cc:1514] Processing DeleteTablet for tablet 20803b8831d943938af1c5eeb6095a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:16 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:54876
I20260809 15:18:16.570408 2424 tablet_replica.cc:330] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: stopping tablet replica
I20260809 15:18:16.570955 2424 raft_consensus.cc:2229] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:16.571534 2424 raft_consensus.cc:2258] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:16.572302 2425 tablet_replica.cc:330] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: stopping tablet replica
I20260809 15:18:16.572693 2425 raft_consensus.cc:2229] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:16.572798 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 1960
I20260809 15:18:16.573215 2425 raft_consensus.cc:2258] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:16.575456 2424 ts_tablet_manager.cc:1905] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:16.583315 2424 ts_tablet_manager.cc:1918] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:16.583491 2424 log.cc:1196] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/20803b8831d943938af1c5eeb6095a67
I20260809 15:18:16.586637 2426 tablet_replica.cc:330] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: stopping tablet replica
I20260809 15:18:16.587085 2426 raft_consensus.cc:2229] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:18:16.587818 2426 raft_consensus.cc:2258] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:16.590328 2426 ts_tablet_manager.cc:1905] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 15:18:16 dist-test-slave-8w9v krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:18:16.597618 2425 ts_tablet_manager.cc:1905] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:16.598483 2426 ts_tablet_manager.cc:1918] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:16.598657 2426 log.cc:1196] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/20803b8831d943938af1c5eeb6095a67
I20260809 15:18:16.599227 2426 ts_tablet_manager.cc:1939] T 20803b8831d943938af1c5eeb6095a67 P 4e8ee879333e4e75a51edc6128fcc7a1: Deleting consensus metadata
I20260809 15:18:16.601336 1878 catalog_manager.cc:4841] TS 4e8ee879333e4e75a51edc6128fcc7a1 (127.15.11.129:40779): tablet 20803b8831d943938af1c5eeb6095a67 (table test-table [id=40625f2c868245dea61799a418fd3b45]) successfully deleted
I20260809 15:18:16.603111 2394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.574132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.15.11.129:40779)
0809 15:18:16.574340 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.574349 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.580679 (+ 6330us) server_negotiation.cc:407] Connection header received
0809 15:18:16.580872 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.580879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.580999 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.581116 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:16.583418 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.585132 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.587776 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.588598 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.588712 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.594720 (+ 6008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:16.594733 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:16.594754 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:16.594798 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:16.599408 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.600375 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.600384 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.600387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.600616 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.601097 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.601105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.601109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.601744 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:16.601991 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.602508 (+ 517us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.602750 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:18:16.603842 2091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:16.610321 2424 ts_tablet_manager.cc:1939] T 20803b8831d943938af1c5eeb6095a67 P 7986775052be4284a5732f8177c488c0: Deleting consensus metadata
I20260809 15:18:16.611924 1879 catalog_manager.cc:4841] TS 7986775052be4284a5732f8177c488c0 (127.15.11.130:42837): tablet 20803b8831d943938af1c5eeb6095a67 (table test-table [id=40625f2c868245dea61799a418fd3b45]) successfully deleted
I20260809 15:18:16.612339 2425 ts_tablet_manager.cc:1918] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:16.612548 2425 log.cc:1196] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/20803b8831d943938af1c5eeb6095a67
I20260809 15:18:16.612994 2425 ts_tablet_manager.cc:1939] T 20803b8831d943938af1c5eeb6095a67 P ac914dc990d14ddd9bf9c62aaeaa6272: Deleting consensus metadata
I20260809 15:18:16.614439 1878 catalog_manager.cc:4841] TS ac914dc990d14ddd9bf9c62aaeaa6272 (127.15.11.131:42547): tablet 20803b8831d943938af1c5eeb6095a67 (table test-table [id=40625f2c868245dea61799a418fd3b45]) successfully deleted
I20260809 15:18:16.705049 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 1960
I20260809 15:18:16.730003 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 2103
Aug 09 15:18:16 dist-test-slave-8w9v krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:18:16.745641 2397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.730856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34198 (local address 127.15.11.130:42837)
0809 15:18:16.731051 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.731056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.731076 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:18:16.731247 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.731251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.731315 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.731409 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:16.732787 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.733544 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.734893 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.735281 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.735456 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.740298 (+ 4842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:16.740314 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:16.740332 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:16.740374 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:16.743042 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.743717 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.743724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.743726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.743869 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.744341 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.744344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.744346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.744732 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:16.744966 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.745230 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.745386 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:18:16.746326 2234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:16.835763 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 2103
I20260809 15:18:16.857793 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 2245
Aug 09 15:18:16 dist-test-slave-8w9v krb5kdc[1847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:18:16.873294 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:16.858576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58248 (local address 127.15.11.131:42547)
0809 15:18:16.858782 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:16.858788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:16.858809 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:18:16.858942 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:16.858946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:16.859020 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:16.859147 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:16.860587 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.861305 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.862588 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:16.862955 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:16.863084 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:16.867704 (+ 4620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:16.867719 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:16.867736 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:16.867768 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:16.870396 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.871094 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.871102 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.871104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.871241 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:16.871660 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:16.871666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:16.871669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:16.872025 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:16.872402 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:16.872728 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 15:18:16.872944 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:18:16.873934 2375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:16.961180 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 2245
I20260809 15:18:16.983028 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 1861
I20260809 15:18:16.984196 1927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:17.095077 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 1861
2026-08-09T15:18:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4545 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T15:18:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:17Z Disabled control of system clock
I20260809 15:18:17.193080 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35941
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:40733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:35941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:18:17.532208 2437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:17.532513 2437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:17.532689 2437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:17.539678 2437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:17.539817 2437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:17.539917 2437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:17.540023 2437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:17.540112 2437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:17.548758 2437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:35941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:17.549984 2437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:17.551932 2437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:17.563789 2445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:17.563856 2443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:17.564023 2442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:17.564563 2437 server_base.cc:1034] running on GCE node
I20260809 15:18:17.565356 2437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:17.566645 2437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:17.567864 2437 hybrid_clock.cc:648] HybridClock initialized: now 1786288697567819 us; error 34 us; skew 500 ppm
I20260809 15:18:17.568269 2437 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:17.574882 2437 webserver.cc:460] Webserver started at http://127.15.11.190:33955/ using document root <none> and password file <none>
I20260809 15:18:17.575572 2437 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:17.575690 2437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:17.576057 2437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:17.578837 2437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "b9818be0fa9c477a9c7300faad767249"
format_stamp: "Formatted at 2026-08-09 15:18:17 on dist-test-slave-8w9v"
server_key: "0804b2ad605ae541e1753fdf4a6fc45d"
server_key_iv: "70cf0fadf9675879e8a300ca7eb55854"
server_key_version: "encryptionkey@0"
I20260809 15:18:17.579656 2437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "b9818be0fa9c477a9c7300faad767249"
format_stamp: "Formatted at 2026-08-09 15:18:17 on dist-test-slave-8w9v"
server_key: "0804b2ad605ae541e1753fdf4a6fc45d"
server_key_iv: "70cf0fadf9675879e8a300ca7eb55854"
server_key_version: "encryptionkey@0"
I20260809 15:18:17.584823 2437 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 15:18:17.588264 2452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:17.589604 2437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:18:17.589802 2437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "b9818be0fa9c477a9c7300faad767249"
format_stamp: "Formatted at 2026-08-09 15:18:17 on dist-test-slave-8w9v"
server_key: "0804b2ad605ae541e1753fdf4a6fc45d"
server_key_iv: "70cf0fadf9675879e8a300ca7eb55854"
server_key_version: "encryptionkey@0"
I20260809 15:18:17.590031 2437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:17.602233 2437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:17.603060 2437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:17.603361 2437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:17.621726 2437 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:35941
I20260809 15:18:17.621750 2505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:35941 every 8 connection(s)
I20260809 15:18:17.623082 2437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:17.626907 2506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:17.629566 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 2437
I20260809 15:18:17.629753 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:17.630398 15406 external_mini_cluster.cc:1428] Setting key 222e98874a70cf6bcb5f15f56045ee77
I20260809 15:18:17.639880 2506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249: Bootstrap starting.
I20260809 15:18:17.643739 2506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:17.644932 2506 log.cc:824] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:17.645169 2509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:17.632434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37334 (local address 127.15.11.190:35941)
0809 15:18:17.633494 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:17.633503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:17.633553 (+ 50us) server_negotiation.cc:407] Connection header received
0809 15:18:17.635204 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:17.635226 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:17.635881 (+ 655us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:17.636400 (+ 519us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:17.637126 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:17.638594 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:17.640607 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:17.641288 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:17.641430 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:17.642670 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:17.642677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:17.642689 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:17.643299 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:17.643390 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:17.643413 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:17.643503 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:17.643780 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 15:18:17.644108 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":385,"threads_started":1}
I20260809 15:18:17.648326 2506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249: No bootstrap required, opened a new log
I20260809 15:18:17.655164 2506 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9818be0fa9c477a9c7300faad767249" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35941 } }
I20260809 15:18:17.655485 2506 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:17.655597 2506 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9818be0fa9c477a9c7300faad767249, State: Initialized, Role: FOLLOWER
I20260809 15:18:17.656186 2506 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9818be0fa9c477a9c7300faad767249" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35941 } }
I20260809 15:18:17.656432 2506 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:17.656558 2506 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:17.656723 2506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:17.658703 2506 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9818be0fa9c477a9c7300faad767249" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35941 } }
I20260809 15:18:17.659183 2506 leader_election.cc:302] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9818be0fa9c477a9c7300faad767249; no voters:
I20260809 15:18:17.659799 2506 leader_election.cc:288] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:17.659960 2511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:17.661111 2511 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [term 1 LEADER]: Becoming Leader. State: Replica: b9818be0fa9c477a9c7300faad767249, State: Running, Role: LEADER
I20260809 15:18:17.661594 2511 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9818be0fa9c477a9c7300faad767249" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35941 } }
I20260809 15:18:17.661916 2506 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:17.664520 2515 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9818be0fa9c477a9c7300faad767249. Latest consensus state: current_term: 1 leader_uuid: "b9818be0fa9c477a9c7300faad767249" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9818be0fa9c477a9c7300faad767249" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35941 } } }
I20260809 15:18:17.664994 2515 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:17.664522 2514 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9818be0fa9c477a9c7300faad767249" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9818be0fa9c477a9c7300faad767249" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35941 } } }
I20260809 15:18:17.665357 2514 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:17.666576 2521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:17.671888 2521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:17.680811 2521 catalog_manager.cc:1347] Generated new cluster ID: a71cfd21f672433c88e7d17af21c5bde
I20260809 15:18:17.680933 2521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:17.711314 2521 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:17.712821 2521 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:17.720513 2521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249: Generated new TSK 0
I20260809 15:18:17.721306 2521 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:17.729691 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35941
--builtin_ntp_servers=127.15.11.148:40733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:18:18.096748 2534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:18.097025 2534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:18.097185 2534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:18.104234 2534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:18.104323 2534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:18.104511 2534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:18:18.113034 2534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:18.114300 2534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:18.116065 2534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:18.129232 2539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:18.129376 2542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:18.129383 2540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:18.130090 2534 server_base.cc:1034] running on GCE node
I20260809 15:18:18.130645 2534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:18.131479 2534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:18.132683 2534 hybrid_clock.cc:648] HybridClock initialized: now 1786288698132631 us; error 44 us; skew 500 ppm
I20260809 15:18:18.133004 2534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:18.135284 2534 webserver.cc:460] Webserver started at http://127.15.11.129:36615/ using document root <none> and password file <none>
I20260809 15:18:18.135946 2534 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:18.136088 2534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:18.136478 2534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:18.139238 2534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "7b5171ecaef4458589c9e6dfa3e0dc18"
format_stamp: "Formatted at 2026-08-09 15:18:18 on dist-test-slave-8w9v"
server_key: "06051abbfe96a2ca51ec4fd68c5aa8a3"
server_key_iv: "5e053bc8940218d3105bf278e51d0614"
server_key_version: "encryptionkey@0"
I20260809 15:18:18.140036 2534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "7b5171ecaef4458589c9e6dfa3e0dc18"
format_stamp: "Formatted at 2026-08-09 15:18:18 on dist-test-slave-8w9v"
server_key: "06051abbfe96a2ca51ec4fd68c5aa8a3"
server_key_iv: "5e053bc8940218d3105bf278e51d0614"
server_key_version: "encryptionkey@0"
I20260809 15:18:18.144930 2534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 15:18:18.148854 2548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:18.150075 2534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:18:18.150270 2534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "7b5171ecaef4458589c9e6dfa3e0dc18"
format_stamp: "Formatted at 2026-08-09 15:18:18 on dist-test-slave-8w9v"
server_key: "06051abbfe96a2ca51ec4fd68c5aa8a3"
server_key_iv: "5e053bc8940218d3105bf278e51d0614"
server_key_version: "encryptionkey@0"
I20260809 15:18:18.150499 2534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:18.167794 2534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:18.168711 2534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:18.169073 2534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:18.170037 2534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:18.171456 2534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:18.171587 2534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:18.171710 2534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:18.171800 2534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:18.205053 2534 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:41151
I20260809 15:18:18.205139 2666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:41151 every 8 connection(s)
I20260809 15:18:18.206398 2534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:18:18.214308 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 2534
I20260809 15:18:18.214520 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:18:18.215112 15406 external_mini_cluster.cc:1428] Setting key 2c2f3091d4bc88e07bc665fca6708289
I20260809 15:18:18.218014 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35941
--builtin_ntp_servers=127.15.11.148:40733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:18:18.219305 2669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:18.208982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:45531 (local address 127.15.11.190:35941)
0809 15:18:18.209414 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:18.209418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:18.210606 (+ 1188us) server_negotiation.cc:407] Connection header received
0809 15:18:18.212403 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:18.212407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:18.212473 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:18.212567 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:18.213537 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:18.214233 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:18.216326 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:18.216773 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:18.216927 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:18.218343 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:18.218352 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:18.218359 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:18.218384 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:18.218432 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:18.218452 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:18.218543 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:18.218738 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 15:18:18.218853 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":311,"thread_start_us":206,"threads_started":1}
I20260809 15:18:18.220005 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:18.209589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35941 (local address 127.15.11.129:45531)
0809 15:18:18.210457 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:18.210491 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:18.212111 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:18.212760 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:18.212767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:18.212840 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:18.213391 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:18.213409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:18.214395 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:18.214401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:18.216194 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:18.216210 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:18.217078 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:18.217085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:18.217336 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:18.218015 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:18:18.218017 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:18.218053 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:18:18.218153 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:18.218549 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:18.218558 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:18.218563 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:18:18.218696 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 15:18:18.218976 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":258,"threads_started":1}
I20260809 15:18:18.223912 2667 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35941
I20260809 15:18:18.224339 2667 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:18.225230 2667 heartbeater.cc:511] Master 127.15.11.190:35941 requested a full tablet report, sending...
I20260809 15:18:18.227348 2470 ts_manager.cc:194] Registered new tserver with Master: 7b5171ecaef4458589c9e6dfa3e0dc18 (127.15.11.129:41151)
I20260809 15:18:18.228881 2470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.129:45531
W20260809 15:18:18.557955 2671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:18.558216 2671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:18.558303 2671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:18.565490 2671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:18.565579 2671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:18.565680 2671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:18:18.574472 2671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:18.575649 2671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:18.577514 2671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:18.590790 2677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:18.590830 2680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:18.590898 2678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:18.596947 2671 server_base.cc:1034] running on GCE node
I20260809 15:18:18.597575 2671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:18.598507 2671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:18.599745 2671 hybrid_clock.cc:648] HybridClock initialized: now 1786288698599690 us; error 59 us; skew 500 ppm
I20260809 15:18:18.600066 2671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:18.602317 2671 webserver.cc:460] Webserver started at http://127.15.11.130:35105/ using document root <none> and password file <none>
I20260809 15:18:18.603003 2671 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:18.603132 2671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:18.603453 2671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:18.606315 2671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "a2aff412f1c34ec7921a244249ee715f"
format_stamp: "Formatted at 2026-08-09 15:18:18 on dist-test-slave-8w9v"
server_key: "99855aaa8c6495d07b91c9e22fa07ed9"
server_key_iv: "ec715585dc235e41894e9e8442734179"
server_key_version: "encryptionkey@0"
I20260809 15:18:18.607087 2671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "a2aff412f1c34ec7921a244249ee715f"
format_stamp: "Formatted at 2026-08-09 15:18:18 on dist-test-slave-8w9v"
server_key: "99855aaa8c6495d07b91c9e22fa07ed9"
server_key_iv: "ec715585dc235e41894e9e8442734179"
server_key_version: "encryptionkey@0"
I20260809 15:18:18.612038 2671 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:18:18.615466 2686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:18.616823 2671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:18:18.617025 2671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "a2aff412f1c34ec7921a244249ee715f"
format_stamp: "Formatted at 2026-08-09 15:18:18 on dist-test-slave-8w9v"
server_key: "99855aaa8c6495d07b91c9e22fa07ed9"
server_key_iv: "ec715585dc235e41894e9e8442734179"
server_key_version: "encryptionkey@0"
I20260809 15:18:18.617238 2671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:18.642540 2671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:18.643338 2671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:18.643613 2671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:18.644543 2671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:18.645905 2671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:18.646018 2671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:18.646087 2671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:18.646145 2671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:18.680275 2671 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:40329
I20260809 15:18:18.680378 2802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:40329 every 8 connection(s)
I20260809 15:18:18.681643 2671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:18:18.688879 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 2671
I20260809 15:18:18.689083 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:18:18.689551 15406 external_mini_cluster.cc:1428] Setting key b3af7080a64ebffa51bbe3c8058a54f3
I20260809 15:18:18.692713 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35941
--builtin_ntp_servers=127.15.11.148:40733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:18:18.694677 2669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:18.684313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:60797 (local address 127.15.11.190:35941)
0809 15:18:18.684505 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:18.684509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:18.685800 (+ 1291us) server_negotiation.cc:407] Connection header received
0809 15:18:18.687459 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:18.687463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:18.687530 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:18.687615 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:18.688820 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:18.689698 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:18.691672 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:18.692054 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:18.692228 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:18.693890 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:18.693894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:18.693897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:18.693916 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:18.693947 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:18.693959 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:18.694029 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:18.694289 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 15:18:18.694414 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:18:18.696026 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:18.684785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35941 (local address 127.15.11.130:60797)
0809 15:18:18.685652 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:18.685687 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:18.687217 (+ 1530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:18.687808 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:18.687815 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:18.687901 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:18.688637 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:18.688663 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:18.689865 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:18.689871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:18.691503 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:18.691516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:18.692318 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:18.692328 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:18.692681 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:18.693500 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:18:18.693506 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:18.693561 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:18:18.693673 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:18.694060 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:18.694073 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:18.694084 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 15:18:18.694262 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 15:18:18.694687 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":264,"threads_started":1}
I20260809 15:18:18.697446 2803 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35941
I20260809 15:18:18.697769 2803 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:18.698660 2803 heartbeater.cc:511] Master 127.15.11.190:35941 requested a full tablet report, sending...
I20260809 15:18:18.700265 2470 ts_manager.cc:194] Registered new tserver with Master: a2aff412f1c34ec7921a244249ee715f (127.15.11.130:40329)
I20260809 15:18:18.701433 2470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.130:60797
W20260809 15:18:19.032331 2806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:19.032603 2806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:19.032846 2806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:19.039836 2806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:19.039939 2806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:19.040107 2806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:18:19.048877 2806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:19.050149 2806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:19.051949 2806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:19.065893 2814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:19.065963 2812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:19.065984 2811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:19.069573 2806 server_base.cc:1034] running on GCE node
I20260809 15:18:19.070097 2806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:19.070952 2806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:19.072212 2806 hybrid_clock.cc:648] HybridClock initialized: now 1786288699072150 us; error 39 us; skew 500 ppm
I20260809 15:18:19.072497 2806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:19.074757 2806 webserver.cc:460] Webserver started at http://127.15.11.131:33999/ using document root <none> and password file <none>
I20260809 15:18:19.075399 2806 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:19.075487 2806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:19.075755 2806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:19.078670 2806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "7b3216376b714d868756d1c927324d0e"
format_stamp: "Formatted at 2026-08-09 15:18:19 on dist-test-slave-8w9v"
server_key: "2af402703cbc0c4b5a708130027b656c"
server_key_iv: "938d26090ba21b50f282542cba43f986"
server_key_version: "encryptionkey@0"
I20260809 15:18:19.079437 2806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "7b3216376b714d868756d1c927324d0e"
format_stamp: "Formatted at 2026-08-09 15:18:19 on dist-test-slave-8w9v"
server_key: "2af402703cbc0c4b5a708130027b656c"
server_key_iv: "938d26090ba21b50f282542cba43f986"
server_key_version: "encryptionkey@0"
I20260809 15:18:19.084599 2806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:18:19.088408 2821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:19.089727 2806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:18:19.089864 2806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "7b3216376b714d868756d1c927324d0e"
format_stamp: "Formatted at 2026-08-09 15:18:19 on dist-test-slave-8w9v"
server_key: "2af402703cbc0c4b5a708130027b656c"
server_key_iv: "938d26090ba21b50f282542cba43f986"
server_key_version: "encryptionkey@0"
I20260809 15:18:19.090046 2806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:19.104034 2806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:19.104856 2806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:19.105118 2806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:19.106036 2806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:19.107399 2806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:19.107486 2806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:19.107570 2806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:19.107626 2806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:19.141662 2806 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:39847
I20260809 15:18:19.141738 2939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:39847 every 8 connection(s)
I20260809 15:18:19.143044 2806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:18:19.143373 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 2806
I20260809 15:18:19.143551 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:18:19.144145 15406 external_mini_cluster.cc:1428] Setting key 00de285a16962661705aab1a28514f46
I20260809 15:18:19.155687 2669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.146005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:50561 (local address 127.15.11.190:35941)
0809 15:18:19.146237 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.146244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.148072 (+ 1828us) server_negotiation.cc:407] Connection header received
0809 15:18:19.149829 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.149832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.149901 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.149982 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:19.150992 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.151696 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.153205 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.153622 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.153781 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.155036 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:19.155041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:19.155044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:19.155062 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:19.155095 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:19.155108 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:19.155178 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.155378 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.155455 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:18:19.156812 2942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.146746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35941 (local address 127.15.11.131:50561)
0809 15:18:19.147911 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.147954 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:19.149601 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:19.150182 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:19.150189 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:19.150270 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:19.150851 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.150870 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.151811 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.151814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.153085 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.153097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.153883 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.153892 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.154095 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.154701 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:18:19.154704 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:19.154746 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:18:19.154836 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:19.155195 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:19.155204 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:19.155209 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:18:19.155351 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 15:18:19.155634 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":393,"threads_started":1}
I20260809 15:18:19.158380 2940 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35941
I20260809 15:18:19.158735 2940 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:19.159633 2940 heartbeater.cc:511] Master 127.15.11.190:35941 requested a full tablet report, sending...
I20260809 15:18:19.161034 2470 ts_manager.cc:194] Registered new tserver with Master: 7b3216376b714d868756d1c927324d0e (127.15.11.131:39847)
I20260809 15:18:19.161855 2470 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.131:50561
I20260809 15:18:19.171505 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:18:19.181212 2669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.175618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37348 (local address 127.15.11.190:35941)
0809 15:18:19.175806 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.175810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.176266 (+ 456us) server_negotiation.cc:407] Connection header received
0809 15:18:19.176409 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.176412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.176483 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.176563 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:19.177049 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.177721 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.179068 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.179488 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.179581 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.180675 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:19.180679 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:19.180682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:19.180698 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:19.180726 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:19.180738 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:19.180803 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.180918 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.180994 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:18:19.184818 2470 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:18:19.191497 2470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:18:19.228807 2953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.204170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58134 (local address 127.15.11.129:41151)
0809 15:18:19.220514 (+ 16344us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.220524 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.220551 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:18:19.220642 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.220651 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.220875 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.221064 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:19.221941 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.223527 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.226179 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.227475 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.227618 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.227947 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.228271 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.228440 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16170,"thread_start_us":16462,"threads_started":1}
I20260809 15:18:19.228852 2954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.204175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:39847 (local address 127.0.0.1:58732)
0809 15:18:19.219871 (+ 15696us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.219891 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:19.220037 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:19.220907 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:19.220916 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:19.220949 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:19.221482 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.221496 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.223612 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.223620 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.225976 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.226007 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.227946 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.227953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.228207 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.228241 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:18:19.228309 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 15:18:19.228455 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15509,"mutex_wait_us":138,"thread_start_us":15326,"threads_started":1}
I20260809 15:18:19.230428 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.204571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.15.11.131:39847)
0809 15:18:19.219469 (+ 14898us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.219479 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.220064 (+ 585us) server_negotiation.cc:407] Connection header received
0809 15:18:19.220207 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.220215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.220487 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.221540 (+ 1053us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:19.221979 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.223476 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.226236 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.227722 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.229554 (+ 1832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.229812 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.229878 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.230040 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14713,"thread_start_us":14560,"threads_started":1}
I20260809 15:18:19.228899 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.204064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41151 (local address 127.0.0.1:58134)
0809 15:18:19.218838 (+ 14774us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.218895 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:19.219079 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:19.221317 (+ 2238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:19.221326 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:19.221355 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:19.221775 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.221789 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.223697 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.223703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.225903 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.225922 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.227709 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.227717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.227930 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.228023 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 15:18:19.228219 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 15:18:19.228434 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14531,"mutex_wait_us":117,"thread_start_us":15233,"threads_started":1}
I20260809 15:18:19.230608 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.220938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:40329 (local address 127.0.0.1:50136)
0809 15:18:19.221580 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.221601 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:19.221878 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:19.222686 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:19.222692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:19.222716 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:19.223131 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.223144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.225046 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.225052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.227331 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.227347 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.229801 (+ 2454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.229808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.229995 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.230025 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:18:19.230118 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 15:18:19.230250 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":68,"thread_start_us":323,"threads_started":1}
I20260809 15:18:19.231648 2667 heartbeater.cc:503] Master 127.15.11.190:35941 was elected leader, sending a full tablet report...
I20260809 15:18:19.231961 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.220366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50136 (local address 127.15.11.130:40329)
0809 15:18:19.220923 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.220933 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.221706 (+ 773us) server_negotiation.cc:407] Connection header received
0809 15:18:19.222046 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.222054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.222279 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.222473 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:19.223324 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.224893 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.227709 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.229216 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.229356 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.229624 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.230223 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.230403 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":285,"threads_started":1}
I20260809 15:18:19.234948 2870 tablet_service.cc:1467] Processing CreateTablet for tablet e2028c6d705c47c68245ad844fe46c54 (DEFAULT_TABLE table=test-table [id=cedab496cbda47d5b2f76438412eb84d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:19.234948 2598 tablet_service.cc:1467] Processing CreateTablet for tablet e2028c6d705c47c68245ad844fe46c54 (DEFAULT_TABLE table=test-table [id=cedab496cbda47d5b2f76438412eb84d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:19.236609 2870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2028c6d705c47c68245ad844fe46c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:19.236609 2598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2028c6d705c47c68245ad844fe46c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:19.238710 2735 tablet_service.cc:1467] Processing CreateTablet for tablet e2028c6d705c47c68245ad844fe46c54 (DEFAULT_TABLE table=test-table [id=cedab496cbda47d5b2f76438412eb84d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:19.239789 2735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2028c6d705c47c68245ad844fe46c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:19.251147 2958 tablet_bootstrap.cc:492] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: Bootstrap starting.
I20260809 15:18:19.254642 2958 tablet_bootstrap.cc:654] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:19.254753 2959 tablet_bootstrap.cc:492] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18: Bootstrap starting.
I20260809 15:18:19.256228 2958 log.cc:824] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:19.256850 2960 tablet_bootstrap.cc:492] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: Bootstrap starting.
I20260809 15:18:19.258467 2959 tablet_bootstrap.cc:654] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:19.259599 2959 log.cc:824] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:19.260680 2960 tablet_bootstrap.cc:654] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:19.261059 2958 tablet_bootstrap.cc:492] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: No bootstrap required, opened a new log
I20260809 15:18:19.261349 2958 ts_tablet_manager.cc:1397] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 15:18:19.261787 2960 log.cc:824] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:19.262660 2959 tablet_bootstrap.cc:492] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18: No bootstrap required, opened a new log
I20260809 15:18:19.262926 2959 ts_tablet_manager.cc:1397] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 15:18:19.264544 2960 tablet_bootstrap.cc:492] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: No bootstrap required, opened a new log
I20260809 15:18:19.264801 2960 ts_tablet_manager.cc:1397] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260809 15:18:19.272027 2960 raft_consensus.cc:348] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } }
I20260809 15:18:19.272544 2960 raft_consensus.cc:374] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:19.272691 2960 raft_consensus.cc:729] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b3216376b714d868756d1c927324d0e, State: Initialized, Role: FOLLOWER
I20260809 15:18:19.272552 2958 raft_consensus.cc:348] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } }
I20260809 15:18:19.272990 2958 raft_consensus.cc:374] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:19.273095 2958 raft_consensus.cc:729] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2aff412f1c34ec7921a244249ee715f, State: Initialized, Role: FOLLOWER
I20260809 15:18:19.273308 2959 raft_consensus.cc:348] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } }
I20260809 15:18:19.273494 2960 consensus_queue.cc:260] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } }
I20260809 15:18:19.273736 2959 raft_consensus.cc:374] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:19.273907 2959 raft_consensus.cc:729] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b5171ecaef4458589c9e6dfa3e0dc18, State: Initialized, Role: FOLLOWER
I20260809 15:18:19.273736 2958 consensus_queue.cc:260] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } }
I20260809 15:18:19.274523 2959 consensus_queue.cc:260] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } }
I20260809 15:18:19.278373 2959 ts_tablet_manager.cc:1428] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260809 15:18:19.279408 2803 heartbeater.cc:503] Master 127.15.11.190:35941 was elected leader, sending a full tablet report...
I20260809 15:18:19.279820 2958 ts_tablet_manager.cc:1428] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
I20260809 15:18:19.280649 2940 heartbeater.cc:503] Master 127.15.11.190:35941 was elected leader, sending a full tablet report...
I20260809 15:18:19.281812 2960 ts_tablet_manager.cc:1428] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: Time spent starting tablet: real 0.017s user 0.010s sys 0.004s
W20260809 15:18:19.395087 2941 tablet.cc:2369] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:18:19.433883 2804 tablet.cc:2369] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:18:19.458806 2668 tablet.cc:2369] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:19.472357 2966 raft_consensus.cc:482] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:18:19.472702 2966 raft_consensus.cc:504] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } }
I20260809 15:18:19.474514 2966 leader_election.cc:288] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7b3216376b714d868756d1c927324d0e (127.15.11.131:39847), a2aff412f1c34ec7921a244249ee715f (127.15.11.130:40329)
I20260809 15:18:19.481709 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.475098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:55359 (local address 127.15.11.130:40329)
0809 15:18:19.475296 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.475305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.476484 (+ 1179us) server_negotiation.cc:407] Connection header received
0809 15:18:19.476700 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.476703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.476775 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.476869 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:19.477505 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.478189 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.479989 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.480793 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.480972 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.481043 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.481325 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.481440 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:18:19.481840 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.474928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:59309 (local address 127.15.11.131:39847)
0809 15:18:19.475147 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.475155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.476546 (+ 1391us) server_negotiation.cc:407] Connection header received
0809 15:18:19.476738 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.476743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.476803 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.476932 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:19.477618 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.478320 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.480060 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.480930 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.481127 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.481195 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.481471 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.481606 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:18:19.482098 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.475403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:40329 (local address 127.15.11.129:55359)
0809 15:18:19.476362 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.476378 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:19.476549 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:19.476884 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:19.476888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:19.476912 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:19.477352 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.477360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.478332 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.478339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.479798 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.479810 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.480984 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.480990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.481152 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.481177 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:18:19.481296 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 15:18:19.481423 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":55,"thread_start_us":407,"threads_started":1}
I20260809 15:18:19.482152 2968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.475440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:39847 (local address 127.15.11.129:59309)
0809 15:18:19.476417 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.476431 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:19.476585 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:19.476932 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:19.476937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:19.476960 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:19.477456 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.477466 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.478455 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.478460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.479853 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:19.479867 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.481133 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.481139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:19.481309 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.481331 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:18:19.481442 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 15:18:19.481557 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"mutex_wait_us":24,"thread_start_us":677,"threads_started":1}
I20260809 15:18:19.483189 2890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2028c6d705c47c68245ad844fe46c54" candidate_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b3216376b714d868756d1c927324d0e" is_pre_election: true
I20260809 15:18:19.483253 2756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2028c6d705c47c68245ad844fe46c54" candidate_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2aff412f1c34ec7921a244249ee715f" is_pre_election: true
I20260809 15:18:19.483701 2890 raft_consensus.cc:2454] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b5171ecaef4458589c9e6dfa3e0dc18 in term 0.
I20260809 15:18:19.483728 2756 raft_consensus.cc:2454] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b5171ecaef4458589c9e6dfa3e0dc18 in term 0.
I20260809 15:18:19.484365 2550 leader_election.cc:302] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b5171ecaef4458589c9e6dfa3e0dc18, a2aff412f1c34ec7921a244249ee715f; no voters:
I20260809 15:18:19.484748 2966 raft_consensus.cc:2781] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:18:19.484892 2966 raft_consensus.cc:482] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:18:19.485024 2966 raft_consensus.cc:3037] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:19.487111 2966 raft_consensus.cc:504] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } }
I20260809 15:18:19.487694 2966 leader_election.cc:288] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [CANDIDATE]: Term 1 election: Requested vote from peers 7b3216376b714d868756d1c927324d0e (127.15.11.131:39847), a2aff412f1c34ec7921a244249ee715f (127.15.11.130:40329)
I20260809 15:18:19.488103 2890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2028c6d705c47c68245ad844fe46c54" candidate_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b3216376b714d868756d1c927324d0e"
I20260809 15:18:19.488336 2890 raft_consensus.cc:3037] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:19.488330 2756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2028c6d705c47c68245ad844fe46c54" candidate_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2aff412f1c34ec7921a244249ee715f"
I20260809 15:18:19.488520 2756 raft_consensus.cc:3037] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:19.490550 2890 raft_consensus.cc:2454] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b5171ecaef4458589c9e6dfa3e0dc18 in term 1.
I20260809 15:18:19.490736 2756 raft_consensus.cc:2454] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b5171ecaef4458589c9e6dfa3e0dc18 in term 1.
I20260809 15:18:19.490959 2551 leader_election.cc:302] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b3216376b714d868756d1c927324d0e, 7b5171ecaef4458589c9e6dfa3e0dc18; no voters:
I20260809 15:18:19.491277 2966 raft_consensus.cc:2781] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:19.491739 2966 raft_consensus.cc:686] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 1 LEADER]: Becoming Leader. State: Replica: 7b5171ecaef4458589c9e6dfa3e0dc18, State: Running, Role: LEADER
I20260809 15:18:19.492249 2966 consensus_queue.cc:237] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } }
I20260809 15:18:19.497244 2468 catalog_manager.cc:5495] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b5171ecaef4458589c9e6dfa3e0dc18 (127.15.11.129). New cstate: current_term: 1 leader_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7b5171ecaef4458589c9e6dfa3e0dc18" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 41151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 } health_report { overall_health: UNKNOWN } } }
I20260809 15:18:19.523821 2953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.518536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.15.11.129:41151)
0809 15:18:19.518705 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.518710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.518815 (+ 105us) server_negotiation.cc:407] Connection header received
0809 15:18:19.518978 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.518981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.519067 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.519185 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:18:19.519509 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.520262 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.521929 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.522326 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.522422 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.522872 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:18:19.523408 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:18:19.523512 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.523584 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.523642 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:18:19.532291 2890 raft_consensus.cc:1261] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [term 1 FOLLOWER]: Refusing update from remote peer 7b5171ecaef4458589c9e6dfa3e0dc18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:19.532308 2756 raft_consensus.cc:1261] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [term 1 FOLLOWER]: Refusing update from remote peer 7b5171ecaef4458589c9e6dfa3e0dc18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:19.533236 2969 consensus_queue.cc:1035] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b3216376b714d868756d1c927324d0e" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 39847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:19.533560 2966 consensus_queue.cc:1035] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2aff412f1c34ec7921a244249ee715f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 40329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:19.557204 2468 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:19.557492 2468 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:19.561249 2468 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9818be0fa9c477a9c7300faad767249: Sending DeleteTablet for 3 replicas of tablet e2028c6d705c47c68245ad844fe46c54
I20260809 15:18:19.562493 2870 tablet_service.cc:1514] Processing DeleteTablet for tablet e2028c6d705c47c68245ad844fe46c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:19 UTC) from {username='slave'} at 127.0.0.1:58732
I20260809 15:18:19.562534 2735 tablet_service.cc:1514] Processing DeleteTablet for tablet e2028c6d705c47c68245ad844fe46c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:19 UTC) from {username='slave'} at 127.0.0.1:50136
I20260809 15:18:19.563525 2598 tablet_service.cc:1514] Processing DeleteTablet for tablet e2028c6d705c47c68245ad844fe46c54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:19 UTC) from {username='slave'} at 127.0.0.1:58134
I20260809 15:18:19.564006 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 2534
I20260809 15:18:19.573344 2985 tablet_replica.cc:330] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: stopping tablet replica
I20260809 15:18:19.573909 2985 raft_consensus.cc:2229] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:19.574517 2985 raft_consensus.cc:2258] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:19.576853 2983 tablet_replica.cc:330] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: stopping tablet replica
I20260809 15:18:19.579272 2985 ts_tablet_manager.cc:1905] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:19.582952 2983 raft_consensus.cc:2229] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:19.583006 2986 tablet_replica.cc:330] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18: stopping tablet replica
I20260809 15:18:19.583441 2986 raft_consensus.cc:2229] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:18:19.583628 2983 raft_consensus.cc:2258] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:19.584230 2986 raft_consensus.cc:2258] T e2028c6d705c47c68245ad844fe46c54 P 7b5171ecaef4458589c9e6dfa3e0dc18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:19.586232 2983 ts_tablet_manager.cc:1905] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:19.586634 2953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.567118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.15.11.129:41151)
0809 15:18:19.567327 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.567337 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.576634 (+ 9297us) server_negotiation.cc:407] Connection header received
0809 15:18:19.577084 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.577089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.577189 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.577278 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:19.577943 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.579103 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.580933 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.581580 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.581675 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.584929 (+ 3254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:19.584943 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:19.584961 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:19.585697 (+ 736us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:19.585756 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:19.585778 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:19.585865 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.586094 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.586212 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:18:19.586669 2985 ts_tablet_manager.cc:1918] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:19.586807 2985 log.cc:1196] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/e2028c6d705c47c68245ad844fe46c54
I20260809 15:18:19.587414 2985 ts_tablet_manager.cc:1939] T e2028c6d705c47c68245ad844fe46c54 P a2aff412f1c34ec7921a244249ee715f: Deleting consensus metadata
I20260809 15:18:19.587878 2661 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:19.589378 2454 catalog_manager.cc:4841] TS a2aff412f1c34ec7921a244249ee715f (127.15.11.130:40329): tablet e2028c6d705c47c68245ad844fe46c54 (table test-table [id=cedab496cbda47d5b2f76438412eb84d]) successfully deleted
I20260809 15:18:19.595465 2983 ts_tablet_manager.cc:1918] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:19.595633 2983 log.cc:1196] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/e2028c6d705c47c68245ad844fe46c54
I20260809 15:18:19.596064 2983 ts_tablet_manager.cc:1939] T e2028c6d705c47c68245ad844fe46c54 P 7b3216376b714d868756d1c927324d0e: Deleting consensus metadata
I20260809 15:18:19.597494 2455 catalog_manager.cc:4841] TS 7b3216376b714d868756d1c927324d0e (127.15.11.131:39847): tablet e2028c6d705c47c68245ad844fe46c54 (table test-table [id=cedab496cbda47d5b2f76438412eb84d]) successfully deleted
I20260809 15:18:19.685004 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 2534
I20260809 15:18:19.713261 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 2671
W20260809 15:18:19.713178 2454 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.11.129:41151 (error 108)
I20260809 15:18:19.715103 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.714570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41151 (local address 127.0.0.1:58146)
0809 15:18:19.714808 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.714932 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260809 15:18:19.715371 2454 catalog_manager.cc:4584] TS 7b5171ecaef4458589c9e6dfa3e0dc18 (127.15.11.129:41151): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2028c6d705c47c68245ad844fe46c54: Network error: Client connection negotiation failed: client connection to 127.15.11.129:41151: connect: Connection refused (error 111)
I20260809 15:18:19.720599 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.714448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50148 (local address 127.15.11.130:40329)
0809 15:18:19.714624 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.714631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.714648 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:18:19.714852 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.714860 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.714960 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.715076 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:19.715620 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.716531 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.717843 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.718240 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.718329 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.719243 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:19.719251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:19.719263 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:19.719797 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:19.719842 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:19.719861 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:19.719948 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.720073 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.720252 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:18:19.721269 2797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:19.768759 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.768339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41151 (local address 127.0.0.1:58154)
0809 15:18:19.768539 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.768616 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 15:18:19.770045 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.769677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41151 (local address 127.0.0.1:58164)
0809 15:18:19.769842 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.769925 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:18:19.813141 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 2671
I20260809 15:18:19.831075 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.830561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41151 (local address 127.0.0.1:58166)
0809 15:18:19.830779 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.830867 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 15:18:19.832165 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.831776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41151 (local address 127.0.0.1:58168)
0809 15:18:19.831941 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.832010 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 15:18:19.834020 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 2806
I20260809 15:18:19.840309 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.834731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.15.11.131:39847)
0809 15:18:19.834886 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:19.834890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:19.834908 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:18:19.835047 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:19.835051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:19.835135 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:19.835218 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:19.835613 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.836392 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.837641 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:19.837999 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:19.838086 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:19.838998 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:19.839007 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:19.839021 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:19.839608 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:19.839652 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:19.839670 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:19.839754 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:19.839862 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 15:18:19.839949 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:18:19.840941 2935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:19.913232 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.912745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41151 (local address 127.0.0.1:58178)
0809 15:18:19.912959 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.913068 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 15:18:19.914439 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:19.913989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:41151 (local address 127.0.0.1:58194)
0809 15:18:19.914228 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:19.914324 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:41151: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260809 15:18:19.927349 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 2806
I20260809 15:18:19.947458 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 2437
I20260809 15:18:19.948529 2501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:20.057938 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 2437
2026-08-09T15:18:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2959 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:20 dist-test-slave-8w9v krb5kdc[2995](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:20 dist-test-slave-8w9v krb5kdc[2995](info): set up 2 sockets
Aug 09 15:18:20 dist-test-slave-8w9v krb5kdc[2995](info): commencing operation
krb5kdc: starting...
W20260809 15:18:21.200176 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:21 dist-test-slave-8w9v krb5kdc[2995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:18:21.487814 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:43425
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:43425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:21.827467 3009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:21.827749 3009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:21.827840 3009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:21.835002 3009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:21.835103 3009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:21.835146 3009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:21.835213 3009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:21.835250 3009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:21.844892 3009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:43425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:43425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:21.846082 3009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:21.847270 3009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:21.859125 3014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:21.859057 3017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:21.859156 3015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:21.860230 3009 server_base.cc:1034] running on GCE node
I20260809 15:18:21.861039 3009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:21.862350 3009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:21.863591 3009 hybrid_clock.cc:648] HybridClock initialized: now 1786288701863533 us; error 42 us; skew 500 ppm
I20260809 15:18:21.863965 3009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:21 dist-test-slave-8w9v krb5kdc[2995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288701, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:21.867022 3009 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:18:21.868642 3009 webserver.cc:460] Webserver started at http://127.15.11.190:45523/ using document root <none> and password file <none>
I20260809 15:18:21.869307 3009 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:21.869468 3009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:21.869772 3009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:21.872797 3009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "02ed19e2446d4c6f957996f9892cb5d7"
format_stamp: "Formatted at 2026-08-09 15:18:21 on dist-test-slave-8w9v"
server_key: "0f3339bdbd02ebd586d50a2f8fd8f834"
server_key_iv: "b181d44e4c20ce6d18f2d1a3b23e29e4"
server_key_version: "encryptionkey@0"
I20260809 15:18:21.873555 3009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "02ed19e2446d4c6f957996f9892cb5d7"
format_stamp: "Formatted at 2026-08-09 15:18:21 on dist-test-slave-8w9v"
server_key: "0f3339bdbd02ebd586d50a2f8fd8f834"
server_key_iv: "b181d44e4c20ce6d18f2d1a3b23e29e4"
server_key_version: "encryptionkey@0"
I20260809 15:18:21.878863 3009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:18:21.882419 3024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:21.883700 3009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 15:18:21.883888 3009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "02ed19e2446d4c6f957996f9892cb5d7"
format_stamp: "Formatted at 2026-08-09 15:18:21 on dist-test-slave-8w9v"
server_key: "0f3339bdbd02ebd586d50a2f8fd8f834"
server_key_iv: "b181d44e4c20ce6d18f2d1a3b23e29e4"
server_key_version: "encryptionkey@0"
I20260809 15:18:21.884083 3009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:21.899391 3009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:21.903218 3009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:21.903514 3009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:21.922447 3009 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:43425
I20260809 15:18:21.922488 3078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:43425 every 8 connection(s)
I20260809 15:18:21.923815 3009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:21.927570 3079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:21.928462 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 3009
I20260809 15:18:21.928670 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:21.929250 15406 external_mini_cluster.cc:1428] Setting key 251913979728c1ffacff2005a5f2d21e
I20260809 15:18:21.940059 3079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7: Bootstrap starting.
I20260809 15:18:21.943252 3079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7: Neither blocks nor log segments found. Creating new log.
Aug 09 15:18:21 dist-test-slave-8w9v krb5kdc[2995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:21.944564 3079 log.cc:824] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:21.947561 3079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7: No bootstrap required, opened a new log
I20260809 15:18:21.952584 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:21.931248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39892 (local address 127.15.11.190:43425)
0809 15:18:21.932023 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:21.932034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:21.932076 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:18:21.934171 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:21.934196 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:21.934571 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:21.934962 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:21.936597 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:21.937914 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:21.939266 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:21.939931 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:21.940134 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:21.944843 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:21.944850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:21.944864 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:21.944905 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:21.948198 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:21.948971 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:21.948976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:21.948982 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:21.949140 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:21.949570 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:21.949573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:21.949575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:21.950083 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:21.950314 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:21.950749 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 15:18:21.951189 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":356,"threads_started":1}
I20260809 15:18:21.956625 3079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ed19e2446d4c6f957996f9892cb5d7" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43425 } }
I20260809 15:18:21.956966 3079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:21.957093 3079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02ed19e2446d4c6f957996f9892cb5d7, State: Initialized, Role: FOLLOWER
I20260809 15:18:21.957619 3079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ed19e2446d4c6f957996f9892cb5d7" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43425 } }
I20260809 15:18:21.957854 3079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:21.958003 3079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:21.958189 3079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:21.960330 3079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ed19e2446d4c6f957996f9892cb5d7" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43425 } }
I20260809 15:18:21.960786 3079 leader_election.cc:302] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02ed19e2446d4c6f957996f9892cb5d7; no voters:
I20260809 15:18:21.961350 3079 leader_election.cc:288] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:21.961478 3085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:21.962172 3085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [term 1 LEADER]: Becoming Leader. State: Replica: 02ed19e2446d4c6f957996f9892cb5d7, State: Running, Role: LEADER
I20260809 15:18:21.962762 3085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ed19e2446d4c6f957996f9892cb5d7" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43425 } }
I20260809 15:18:21.963357 3079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:21.966113 3087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02ed19e2446d4c6f957996f9892cb5d7. Latest consensus state: current_term: 1 leader_uuid: "02ed19e2446d4c6f957996f9892cb5d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ed19e2446d4c6f957996f9892cb5d7" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43425 } } }
I20260809 15:18:21.966576 3087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:21.966481 3086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02ed19e2446d4c6f957996f9892cb5d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02ed19e2446d4c6f957996f9892cb5d7" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43425 } } }
I20260809 15:18:21.966800 3086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:21.968070 3092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:21.974299 3092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:21.978492 3104 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 15:18:21.983949 3092 catalog_manager.cc:1347] Generated new cluster ID: 4d54b1be7a0a432bb76b429d53668a4c
I20260809 15:18:21.984140 3092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:22.013664 3092 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:22.015156 3092 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:22.027715 3092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7: Generated new TSK 0
I20260809 15:18:22.028733 3092 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:18:22.101920 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43425
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:22.447635 3108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:22.448040 3108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:22.448228 3108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:22.455955 3108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:22.456137 3108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:22.456346 3108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:18:22.466696 3108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:22.468247 3108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:22.469756 3108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:22.483026 3113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:22.483060 3116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:22.483125 3114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:22.483902 3108 server_base.cc:1034] running on GCE node
I20260809 15:18:22.484545 3108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:22.485507 3108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:22.486732 3108 hybrid_clock.cc:648] HybridClock initialized: now 1786288702486683 us; error 45 us; skew 500 ppm
I20260809 15:18:22.487057 3108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:22 dist-test-slave-8w9v krb5kdc[2995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288702, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:22.490373 3108 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:18:22.491786 3108 webserver.cc:460] Webserver started at http://127.15.11.129:37235/ using document root <none> and password file <none>
I20260809 15:18:22.492506 3108 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:22.492610 3108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:22.492863 3108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:22.495625 3108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "c7bbb5a7fb334092a4117b1b553676ad"
format_stamp: "Formatted at 2026-08-09 15:18:22 on dist-test-slave-8w9v"
server_key: "48c2d0edb5072cddfb8a41cd9fb7b4cc"
server_key_iv: "b9f7fd82056321bc06affafb252b1df5"
server_key_version: "encryptionkey@0"
I20260809 15:18:22.496440 3108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "c7bbb5a7fb334092a4117b1b553676ad"
format_stamp: "Formatted at 2026-08-09 15:18:22 on dist-test-slave-8w9v"
server_key: "48c2d0edb5072cddfb8a41cd9fb7b4cc"
server_key_iv: "b9f7fd82056321bc06affafb252b1df5"
server_key_version: "encryptionkey@0"
I20260809 15:18:22.501327 3108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:18:22.504575 3124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:22.505795 3108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:18:22.505973 3108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "c7bbb5a7fb334092a4117b1b553676ad"
format_stamp: "Formatted at 2026-08-09 15:18:22 on dist-test-slave-8w9v"
server_key: "48c2d0edb5072cddfb8a41cd9fb7b4cc"
server_key_iv: "b9f7fd82056321bc06affafb252b1df5"
server_key_version: "encryptionkey@0"
I20260809 15:18:22.506137 3108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:22.523375 3108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:22.527653 3108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:22.527957 3108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:22.528959 3108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:22.530426 3108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:22.530499 3108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:22.530678 3108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:22.530733 3108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:22.564740 3108 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:44381
I20260809 15:18:22.564805 3241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:44381 every 8 connection(s)
I20260809 15:18:22.566085 3108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:18:22.572739 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 3108
I20260809 15:18:22.572942 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:18:22.573421 15406 external_mini_cluster.cc:1428] Setting key 62e8fac79f2d06f7d1a06be7b59d9ee6
Aug 09 15:18:22 dist-test-slave-8w9v krb5kdc[2995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288702, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:22.593657 3244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:22.568595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:50069 (local address 127.15.11.190:43425)
0809 15:18:22.569090 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:22.569095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:22.570246 (+ 1151us) server_negotiation.cc:407] Connection header received
0809 15:18:22.572179 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:22.572185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:22.572285 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:22.572440 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:22.574154 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:22.575010 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:22.576720 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:22.577493 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:22.577642 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:22.586471 (+ 8829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:22.586479 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:22.586486 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:22.586531 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:22.590319 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:22.591054 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:22.591062 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:22.591068 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:22.591217 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:22.591616 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:22.591624 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:22.591630 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:22.592020 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:22.592198 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:22.592895 (+ 697us) server_negotiation.cc:299] Negotiation successful
0809 15:18:22.593170 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":241,"threads_started":1}
I20260809 15:18:22.594218 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:22.569237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43425 (local address 127.15.11.129:50069)
0809 15:18:22.570091 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:22.570129 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:22.571852 (+ 1723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:22.572650 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:22.572658 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:22.573290 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:22.573939 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:22.573966 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:22.575178 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:22.575183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:22.576578 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:22.576590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:22.577741 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:22.577746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:22.577936 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:22.583198 (+ 5262us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:18:22.583237 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:22.586236 (+ 2999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:22.590478 (+ 4242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:22.590485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:22.590497 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:22.590936 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:22.591323 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:22.591326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:22.591328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:22.591494 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:22.592194 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:22.592200 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:22.592511 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 15:18:22.592733 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 15:18:22.593111 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":303,"threads_started":1}
I20260809 15:18:22.595693 3242 heartbeater.cc:347] Connected to a master server at 127.15.11.190:43425
I20260809 15:18:22.596014 3242 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:22.596799 3242 heartbeater.cc:511] Master 127.15.11.190:43425 requested a full tablet report, sending...
I20260809 15:18:22.598968 3041 ts_manager.cc:194] Registered new tserver with Master: c7bbb5a7fb334092a4117b1b553676ad (127.15.11.129:44381)
I20260809 15:18:22.600461 3041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:50069
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:18:22.640478 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43425
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:22.979084 3248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:22.979370 3248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:22.979471 3248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:22.986620 3248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:22.986744 3248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:22.986863 3248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:18:22.996591 3248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:22.997956 3248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:22.999279 3248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:23.012676 3254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:23.012845 3256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:23.012809 3253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:23.013828 3248 server_base.cc:1034] running on GCE node
I20260809 15:18:23.014514 3248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:23.015491 3248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:23.016719 3248 hybrid_clock.cc:648] HybridClock initialized: now 1786288703016662 us; error 49 us; skew 500 ppm
I20260809 15:18:23.017042 3248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:23 dist-test-slave-8w9v krb5kdc[2995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288703, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:23.020303 3248 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:18:23.021790 3248 webserver.cc:460] Webserver started at http://127.15.11.130:39647/ using document root <none> and password file <none>
I20260809 15:18:23.022492 3248 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:23.022644 3248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:23.022971 3248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:23.025965 3248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "d1510e0e9dce4619af3e57f9cc9a830f"
format_stamp: "Formatted at 2026-08-09 15:18:23 on dist-test-slave-8w9v"
server_key: "c98fd5d49f4a186e1966f218ffd4c7b0"
server_key_iv: "fc7c2d66f21b1260362b655409a95955"
server_key_version: "encryptionkey@0"
I20260809 15:18:23.026770 3248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "d1510e0e9dce4619af3e57f9cc9a830f"
format_stamp: "Formatted at 2026-08-09 15:18:23 on dist-test-slave-8w9v"
server_key: "c98fd5d49f4a186e1966f218ffd4c7b0"
server_key_iv: "fc7c2d66f21b1260362b655409a95955"
server_key_version: "encryptionkey@0"
I20260809 15:18:23.032094 3248 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 15:18:23.035640 3263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:23.037108 3248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:18:23.037245 3248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "d1510e0e9dce4619af3e57f9cc9a830f"
format_stamp: "Formatted at 2026-08-09 15:18:23 on dist-test-slave-8w9v"
server_key: "c98fd5d49f4a186e1966f218ffd4c7b0"
server_key_iv: "fc7c2d66f21b1260362b655409a95955"
server_key_version: "encryptionkey@0"
I20260809 15:18:23.037415 3248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:23.055794 3248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:23.059729 3248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:23.059959 3248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:23.060942 3248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:23.062204 3248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:23.062275 3248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:23.062359 3248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:23.062412 3248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:23.096386 3248 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:43147
I20260809 15:18:23.096424 3380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:43147 every 8 connection(s)
I20260809 15:18:23.097678 3248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:18:23.099273 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 3248
I20260809 15:18:23.099486 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:18:23.100096 15406 external_mini_cluster.cc:1428] Setting key e3a5fffeb5603244334cd832d5feed9a
Aug 09 15:18:23 dist-test-slave-8w9v krb5kdc[2995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288703, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:23.119031 3383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.100475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:58821 (local address 127.15.11.190:43425)
0809 15:18:23.101320 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.101326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.102036 (+ 710us) server_negotiation.cc:407] Connection header received
0809 15:18:23.103954 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.103959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.104059 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.104198 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:23.105671 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.106372 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.107930 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.108420 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.108555 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.112505 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:23.112513 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:23.112521 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:23.112569 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:23.115808 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:23.116809 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:23.116813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:23.116815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:23.116955 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:23.117433 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:23.117435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:23.117437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:23.117684 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:23.117801 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.118613 (+ 812us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.118771 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":725,"thread_start_us":624,"threads_started":1}
I20260809 15:18:23.120647 3384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.101070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43425 (local address 127.15.11.130:58821)
0809 15:18:23.101876 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:23.101923 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:23.103678 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:23.104438 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:23.104447 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:23.104942 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:23.105498 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.105515 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.106535 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.106545 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.107771 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.107783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.108687 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.108695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.108923 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.109667 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:18:23.109697 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:23.112260 (+ 2563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:23.115947 (+ 3687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:23.115957 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:23.115975 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:23.116701 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:23.117067 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:23.117074 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:23.117080 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:23.117317 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:23.117807 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:23.117817 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:23.118239 (+ 422us) client_negotiation.cc:769] Sending connection context
0809 15:18:23.118555 (+ 316us) client_negotiation.cc:240] Negotiation successful
0809 15:18:23.119146 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":404,"threads_started":1}
I20260809 15:18:23.122128 3381 heartbeater.cc:347] Connected to a master server at 127.15.11.190:43425
I20260809 15:18:23.122435 3381 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:23.123180 3381 heartbeater.cc:511] Master 127.15.11.190:43425 requested a full tablet report, sending...
I20260809 15:18:23.124698 3041 ts_manager.cc:194] Registered new tserver with Master: d1510e0e9dce4619af3e57f9cc9a830f (127.15.11.130:43147)
I20260809 15:18:23.125505 3041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:58821
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:18:23.166669 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43425
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:23.522756 3387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:23.523025 3387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:23.523243 3387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:23.529949 3387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:23.530071 3387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:23.530252 3387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:18:23.539417 3387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:43425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:23.540732 3387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:23.542105 3387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:23.554618 3392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:23.554666 3393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:23.554708 3396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:23.555706 3387 server_base.cc:1034] running on GCE node
I20260809 15:18:23.556432 3387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:23.557271 3387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:23.558501 3387 hybrid_clock.cc:648] HybridClock initialized: now 1786288703558442 us; error 42 us; skew 500 ppm
I20260809 15:18:23.558810 3387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:23 dist-test-slave-8w9v krb5kdc[2995](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288703, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:23.561986 3387 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:18:23.563427 3387 webserver.cc:460] Webserver started at http://127.15.11.131:46703/ using document root <none> and password file <none>
I20260809 15:18:23.564193 3387 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:23.564347 3387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:23.564810 3387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:23.567508 3387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "081f48bae01247d4b49ae490be5b58d6"
format_stamp: "Formatted at 2026-08-09 15:18:23 on dist-test-slave-8w9v"
server_key: "4e2893311118a3db2f3534c19430b5b3"
server_key_iv: "8bd033708048add81aaf5d203ddddcef"
server_key_version: "encryptionkey@0"
I20260809 15:18:23.568418 3387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "081f48bae01247d4b49ae490be5b58d6"
format_stamp: "Formatted at 2026-08-09 15:18:23 on dist-test-slave-8w9v"
server_key: "4e2893311118a3db2f3534c19430b5b3"
server_key_iv: "8bd033708048add81aaf5d203ddddcef"
server_key_version: "encryptionkey@0"
I20260809 15:18:23.573611 3387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:18:23.576996 3403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:23.578231 3387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:18:23.578399 3387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "081f48bae01247d4b49ae490be5b58d6"
format_stamp: "Formatted at 2026-08-09 15:18:23 on dist-test-slave-8w9v"
server_key: "4e2893311118a3db2f3534c19430b5b3"
server_key_iv: "8bd033708048add81aaf5d203ddddcef"
server_key_version: "encryptionkey@0"
I20260809 15:18:23.578589 3387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:23.595541 3387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:23.599568 3387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:23.599844 3387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:23.600797 3387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:23.602039 3387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:23.602144 3387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:23.602296 3387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:23.602368 3387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:23.603101 3242 heartbeater.cc:503] Master 127.15.11.190:43425 was elected leader, sending a full tablet report...
I20260809 15:18:23.636627 3387 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:33861
I20260809 15:18:23.636675 3523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:33861 every 8 connection(s)
I20260809 15:18:23.637956 3387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:18:23.644646 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 3387
I20260809 15:18:23.644825 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:18:23.645311 15406 external_mini_cluster.cc:1428] Setting key 6402b91b3b3289f1051f1eebbe1a9f99
Aug 09 15:18:23 dist-test-slave-8w9v krb5kdc[2995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288703, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:23.657806 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.640546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:53845 (local address 127.15.11.190:43425)
0809 15:18:23.641067 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.641071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.642115 (+ 1044us) server_negotiation.cc:407] Connection header received
0809 15:18:23.643904 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.643910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.644010 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.644181 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:23.645868 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.646844 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.648586 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.648988 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.649119 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.652438 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:23.652443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:23.652446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:23.652476 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:23.654771 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:23.655462 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:23.655467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:23.655470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:23.655591 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:23.656011 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:23.656015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:23.656018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:23.656358 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:23.656531 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.657185 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.657425 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":249,"threads_started":1}
I20260809 15:18:23.658731 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.641152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:43425 (local address 127.15.11.131:53845)
0809 15:18:23.641962 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:23.642008 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:23.643643 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:23.644434 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:23.644442 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:23.645059 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:23.645688 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.645706 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.647011 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.647015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.648439 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.648452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.649211 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.649220 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.649441 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.650130 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:18:23.650163 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:23.652227 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:23.654913 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:23.654921 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:23.654934 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:23.655324 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:23.655697 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:23.655702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:23.655705 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:23.655884 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:23.656495 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:23.656503 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:23.656809 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 15:18:23.657071 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 15:18:23.657553 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":272,"threads_started":1}
I20260809 15:18:23.660349 3524 heartbeater.cc:347] Connected to a master server at 127.15.11.190:43425
I20260809 15:18:23.660606 3524 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:23.661257 3524 heartbeater.cc:511] Master 127.15.11.190:43425 requested a full tablet report, sending...
I20260809 15:18:23.662549 3041 ts_manager.cc:194] Registered new tserver with Master: 081f48bae01247d4b49ae490be5b58d6 (127.15.11.131:33861)
I20260809 15:18:23.663265 3041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:53845
I20260809 15:18:23.672356 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:18:23.689002 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.676344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.15.11.190:43425)
0809 15:18:23.676565 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.676574 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.677003 (+ 429us) server_negotiation.cc:407] Connection header received
0809 15:18:23.677132 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.677136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.677205 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.677280 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:23.678582 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.679246 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.680736 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.681111 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.681198 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.684011 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:23.684015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:23.684018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:23.684046 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:23.686386 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:23.687094 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:23.687101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:23.687108 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:23.687213 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:23.687614 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:23.687617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:23.687619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:23.687889 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:23.688082 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.688472 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.688653 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:18:23.692677 3041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:18:23.699424 3041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:18:23.738638 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.711646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.0.0.1:42742)
0809 15:18:23.726809 (+ 15163us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:23.726840 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:23.727037 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:23.728075 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:23.728085 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:23.728129 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:23.728784 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.728800 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.731250 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.731258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.734243 (+ 2985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.734257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.737325 (+ 3068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.737336 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.737643 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.737699 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 15:18:23.737796 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 15:18:23.737976 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14912,"mutex_wait_us":14,"spinlock_wait_cycles":3840,"thread_start_us":15564,"threads_started":1}
I20260809 15:18:23.738806 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.712445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44190 (local address 127.15.11.130:43147)
0809 15:18:23.726319 (+ 13874us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.726328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.726974 (+ 646us) server_negotiation.cc:407] Connection header received
0809 15:18:23.727270 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.727279 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.727554 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.727772 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:23.729080 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.730727 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.734586 (+ 3859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.736345 (+ 1759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.736460 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.736719 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.738154 (+ 1435us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.738390 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13728,"thread_start_us":13622,"threads_started":1}
I20260809 15:18:23.739615 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.712362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:43147 (local address 127.0.0.1:44190)
0809 15:18:23.726806 (+ 14444us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:23.726838 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:23.727037 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:23.728155 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:23.728163 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:23.728200 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:23.728819 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.728838 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.730964 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.730972 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.734352 (+ 3380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.734372 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.736990 (+ 2618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.736999 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.737334 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.737451 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 15:18:23.737624 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 15:18:23.737896 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14284,"mutex_wait_us":48,"spinlock_wait_cycles":4864,"thread_start_us":14128,"threads_started":1}
I20260809 15:18:23.739962 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.711937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42742 (local address 127.15.11.129:44381)
0809 15:18:23.724991 (+ 13054us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.725002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.727229 (+ 2227us) server_negotiation.cc:407] Connection header received
0809 15:18:23.727310 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.727315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.727544 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.727737 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:23.729021 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.731041 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.734464 (+ 3423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.736532 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.736701 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.736995 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.739131 (+ 2136us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.739383 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12836,"thread_start_us":13732,"threads_started":1}
I20260809 15:18:23.744637 3313 tablet_service.cc:1467] Processing CreateTablet for tablet 8b2d02be8c3c463883579b66538821ae (DEFAULT_TABLE table=test-table [id=082be13d41f44903948c33fba865ccdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:23.745354 3173 tablet_service.cc:1467] Processing CreateTablet for tablet 8b2d02be8c3c463883579b66538821ae (DEFAULT_TABLE table=test-table [id=082be13d41f44903948c33fba865ccdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:23.746326 3313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b2d02be8c3c463883579b66538821ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:23.746776 3173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b2d02be8c3c463883579b66538821ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:23.748857 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.727552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:33861 (local address 127.0.0.1:53510)
0809 15:18:23.739898 (+ 12346us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:23.739923 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:23.740039 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:23.741826 (+ 1787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:23.741834 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:23.741865 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:23.742017 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.742028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.743944 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.743949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.745855 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.745872 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.748075 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.748080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.748287 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.748321 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:18:23.748428 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 15:18:23.748574 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12204,"thread_start_us":14169,"threads_started":1}
I20260809 15:18:23.749768 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.727864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53510 (local address 127.15.11.131:33861)
0809 15:18:23.741029 (+ 13165us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.741042 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.741071 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:18:23.741150 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.741155 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.741326 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.741547 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:23.742211 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.743803 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.746112 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.747617 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.747745 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.747971 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.749271 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.749443 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12933,"thread_start_us":12777,"threads_started":1}
I20260809 15:18:23.755484 3457 tablet_service.cc:1467] Processing CreateTablet for tablet 8b2d02be8c3c463883579b66538821ae (DEFAULT_TABLE table=test-table [id=082be13d41f44903948c33fba865ccdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:23.757009 3457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b2d02be8c3c463883579b66538821ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:23.759594 3545 tablet_bootstrap.cc:492] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: Bootstrap starting.
I20260809 15:18:23.761561 3546 tablet_bootstrap.cc:492] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: Bootstrap starting.
I20260809 15:18:23.763549 3545 tablet_bootstrap.cc:654] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:23.765009 3545 log.cc:824] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:23.765204 3546 tablet_bootstrap.cc:654] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:23.766554 3546 log.cc:824] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:23.770447 3549 tablet_bootstrap.cc:492] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: Bootstrap starting.
I20260809 15:18:23.770969 3545 tablet_bootstrap.cc:492] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: No bootstrap required, opened a new log
I20260809 15:18:23.771246 3545 ts_tablet_manager.cc:1397] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20260809 15:18:23.771252 3546 tablet_bootstrap.cc:492] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: No bootstrap required, opened a new log
I20260809 15:18:23.771513 3546 ts_tablet_manager.cc:1397] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260809 15:18:23.774235 3549 tablet_bootstrap.cc:654] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:23.775429 3549 log.cc:824] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:23.779157 3546 raft_consensus.cc:348] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } }
I20260809 15:18:23.779505 3546 raft_consensus.cc:374] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:23.779624 3546 raft_consensus.cc:729] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bbb5a7fb334092a4117b1b553676ad, State: Initialized, Role: FOLLOWER
I20260809 15:18:23.780314 3546 consensus_queue.cc:260] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } }
I20260809 15:18:23.782209 3546 ts_tablet_manager.cc:1428] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: Time spent starting tablet: real 0.011s user 0.010s sys 0.001s
I20260809 15:18:23.782574 3545 raft_consensus.cc:348] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } }
I20260809 15:18:23.782929 3549 tablet_bootstrap.cc:492] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: No bootstrap required, opened a new log
I20260809 15:18:23.783028 3545 raft_consensus.cc:374] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:23.783145 3545 raft_consensus.cc:729] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1510e0e9dce4619af3e57f9cc9a830f, State: Initialized, Role: FOLLOWER
I20260809 15:18:23.783182 3549 ts_tablet_manager.cc:1397] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260809 15:18:23.783697 3545 consensus_queue.cc:260] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } }
I20260809 15:18:23.784789 3381 heartbeater.cc:503] Master 127.15.11.190:43425 was elected leader, sending a full tablet report...
I20260809 15:18:23.785435 3545 ts_tablet_manager.cc:1428] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 15:18:23.791805 3549 raft_consensus.cc:348] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } }
I20260809 15:18:23.792189 3549 raft_consensus.cc:374] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:23.792311 3549 raft_consensus.cc:729] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 081f48bae01247d4b49ae490be5b58d6, State: Initialized, Role: FOLLOWER
I20260809 15:18:23.792820 3549 consensus_queue.cc:260] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } }
I20260809 15:18:23.793759 3524 heartbeater.cc:503] Master 127.15.11.190:43425 was elected leader, sending a full tablet report...
I20260809 15:18:23.794134 3549 ts_tablet_manager.cc:1428] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
W20260809 15:18:23.818663 3243 tablet.cc:2369] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:23.833961 3554 raft_consensus.cc:482] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:18:23.834213 3554 raft_consensus.cc:504] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } }
I20260809 15:18:23.836282 3554 leader_election.cc:288] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7bbb5a7fb334092a4117b1b553676ad (127.15.11.129:44381), d1510e0e9dce4619af3e57f9cc9a830f (127.15.11.130:43147)
I20260809 15:18:23.843272 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.836668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.15.11.131:51881)
0809 15:18:23.836972 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:23.836986 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:23.837145 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:23.837620 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:23.837627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:23.837655 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:23.837848 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.837856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.839147 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.839151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.841143 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.841159 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.842624 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.842628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.842752 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.842776 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:18:23.842843 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 15:18:23.842925 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":47}
I20260809 15:18:23.843288 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.836593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:51881 (local address 127.15.11.129:44381)
0809 15:18:23.836848 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.836857 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.837229 (+ 372us) server_negotiation.cc:407] Connection header received
0809 15:18:23.837378 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.837386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.837476 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.837578 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:23.838005 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.839006 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.841373 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.842481 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.842630 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.842703 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.842906 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.843053 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260809 15:18:23.843914 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.837116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:44219 (local address 127.15.11.130:43147)
0809 15:18:23.837295 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.837303 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.837670 (+ 367us) server_negotiation.cc:407] Connection header received
0809 15:18:23.837787 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.837791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.837849 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.837974 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:23.838549 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.839447 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.841697 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.842906 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.843064 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.843144 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.843515 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.843629 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:18:23.844267 3194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2d02be8c3c463883579b66538821ae" candidate_uuid: "081f48bae01247d4b49ae490be5b58d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bbb5a7fb334092a4117b1b553676ad" is_pre_election: true
I20260809 15:18:23.844625 3555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.836710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:43147 (local address 127.15.11.131:44219)
0809 15:18:23.837553 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:23.837571 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:23.837676 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:23.837985 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:23.837990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:23.838016 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:23.838389 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.838402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.839614 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.839619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.841557 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:23.841568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.843137 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.843144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:23.843307 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.843345 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:18:23.844091 (+ 746us) client_negotiation.cc:240] Negotiation successful
0809 15:18:23.844251 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"mutex_wait_us":1,"thread_start_us":590,"threads_started":1}
I20260809 15:18:23.844844 3194 raft_consensus.cc:2454] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 081f48bae01247d4b49ae490be5b58d6 in term 0.
I20260809 15:18:23.845600 3335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2d02be8c3c463883579b66538821ae" candidate_uuid: "081f48bae01247d4b49ae490be5b58d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" is_pre_election: true
I20260809 15:18:23.846002 3335 raft_consensus.cc:2454] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 081f48bae01247d4b49ae490be5b58d6 in term 0.
I20260809 15:18:23.846644 3404 leader_election.cc:302] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 081f48bae01247d4b49ae490be5b58d6, d1510e0e9dce4619af3e57f9cc9a830f; no voters:
I20260809 15:18:23.847129 3554 raft_consensus.cc:2781] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:18:23.847297 3554 raft_consensus.cc:482] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:18:23.847411 3554 raft_consensus.cc:3037] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:18:23.849561 3382 tablet.cc:2369] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:23.849771 3554 raft_consensus.cc:504] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } }
I20260809 15:18:23.850425 3554 leader_election.cc:288] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [CANDIDATE]: Term 1 election: Requested vote from peers c7bbb5a7fb334092a4117b1b553676ad (127.15.11.129:44381), d1510e0e9dce4619af3e57f9cc9a830f (127.15.11.130:43147)
I20260809 15:18:23.850833 3194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2d02be8c3c463883579b66538821ae" candidate_uuid: "081f48bae01247d4b49ae490be5b58d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7bbb5a7fb334092a4117b1b553676ad"
I20260809 15:18:23.850935 3335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2d02be8c3c463883579b66538821ae" candidate_uuid: "081f48bae01247d4b49ae490be5b58d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1510e0e9dce4619af3e57f9cc9a830f"
I20260809 15:18:23.851066 3194 raft_consensus.cc:3037] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:23.851120 3335 raft_consensus.cc:3037] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:23.854308 3194 raft_consensus.cc:2454] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 081f48bae01247d4b49ae490be5b58d6 in term 1.
I20260809 15:18:23.854308 3335 raft_consensus.cc:2454] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 081f48bae01247d4b49ae490be5b58d6 in term 1.
I20260809 15:18:23.854739 3404 leader_election.cc:302] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 081f48bae01247d4b49ae490be5b58d6, d1510e0e9dce4619af3e57f9cc9a830f; no voters:
I20260809 15:18:23.855033 3554 raft_consensus.cc:2781] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:23.855544 3554 raft_consensus.cc:686] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 1 LEADER]: Becoming Leader. State: Replica: 081f48bae01247d4b49ae490be5b58d6, State: Running, Role: LEADER
I20260809 15:18:23.856078 3554 consensus_queue.cc:237] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } }
I20260809 15:18:23.860769 3040 catalog_manager.cc:5495] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 081f48bae01247d4b49ae490be5b58d6 (127.15.11.131). New cstate: current_term: 1 leader_uuid: "081f48bae01247d4b49ae490be5b58d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "081f48bae01247d4b49ae490be5b58d6" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 } health_report { overall_health: UNKNOWN } } }
I20260809 15:18:23.877310 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.871531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.15.11.131:33861)
0809 15:18:23.871706 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.871712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.871824 (+ 112us) server_negotiation.cc:407] Connection header received
0809 15:18:23.872053 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.872057 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.872169 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.872319 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:18:23.872754 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.873478 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.875069 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.875472 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.875607 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.876252 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:18:23.876706 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:18:23.876917 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.877026 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.877069 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:18:23.885218 3335 raft_consensus.cc:1261] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [term 1 FOLLOWER]: Refusing update from remote peer 081f48bae01247d4b49ae490be5b58d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:23.885243 3194 raft_consensus.cc:1261] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [term 1 FOLLOWER]: Refusing update from remote peer 081f48bae01247d4b49ae490be5b58d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:23.886168 3554 consensus_queue.cc:1035] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7bbb5a7fb334092a4117b1b553676ad" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 44381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:23.886444 3556 consensus_queue.cc:1035] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1510e0e9dce4619af3e57f9cc9a830f" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:18:23.889596 3526 tablet.cc:2369] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:23.896205 3559 mvcc.cc:205] Tried to move back new op lower bound from 7316638531104223232 to 7316638530998628352. Current Snapshot: MvccSnapshot[applied={T|T < 7316638531104223232}]
I20260809 15:18:23.899892 3561 mvcc.cc:205] Tried to move back new op lower bound from 7316638531104223232 to 7316638530998628352. Current Snapshot: MvccSnapshot[applied={T|T < 7316638531104223232 or (T in {7316638531104223232})}]
I20260809 15:18:23.913919 3041 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:23.914212 3041 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:23.917685 3041 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02ed19e2446d4c6f957996f9892cb5d7: Sending DeleteTablet for 3 replicas of tablet 8b2d02be8c3c463883579b66538821ae
I20260809 15:18:23.918836 3313 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b2d02be8c3c463883579b66538821ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:23 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:44190
I20260809 15:18:23.919009 3173 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b2d02be8c3c463883579b66538821ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:23 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:42742
I20260809 15:18:23.919659 3457 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b2d02be8c3c463883579b66538821ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:23 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:53510
I20260809 15:18:23.928467 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 3108
I20260809 15:18:23.934166 3570 tablet_replica.cc:330] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: stopping tablet replica
I20260809 15:18:23.934727 3570 raft_consensus.cc:2229] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:23.935366 3570 raft_consensus.cc:2258] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:23.937862 3570 ts_tablet_manager.cc:1905] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:23.945087 3572 tablet_replica.cc:330] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: stopping tablet replica
I20260809 15:18:23.945363 3570 ts_tablet_manager.cc:1918] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:23.945561 3570 log.cc:1196] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/8b2d02be8c3c463883579b66538821ae
I20260809 15:18:23.945792 3572 raft_consensus.cc:2229] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:18:23.946252 3570 ts_tablet_manager.cc:1939] T 8b2d02be8c3c463883579b66538821ae P d1510e0e9dce4619af3e57f9cc9a830f: Deleting consensus metadata
I20260809 15:18:23.946965 3572 raft_consensus.cc:2258] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:23.947206 3571 tablet_replica.cc:330] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: stopping tablet replica
I20260809 15:18:23.947700 3571 raft_consensus.cc:2229] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:23.948488 3025 catalog_manager.cc:4841] TS d1510e0e9dce4619af3e57f9cc9a830f (127.15.11.130:43147): tablet 8b2d02be8c3c463883579b66538821ae (table test-table [id=082be13d41f44903948c33fba865ccdf]) successfully deleted
I20260809 15:18:23.950042 3572 ts_tablet_manager.cc:1905] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:23.948592 3571 raft_consensus.cc:2258] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:23.957897 3572 ts_tablet_manager.cc:1918] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:23.958130 3572 log.cc:1196] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/8b2d02be8c3c463883579b66538821ae
I20260809 15:18:23.958761 3572 ts_tablet_manager.cc:1939] T 8b2d02be8c3c463883579b66538821ae P 081f48bae01247d4b49ae490be5b58d6: Deleting consensus metadata
Aug 09 15:18:23 dist-test-slave-8w9v krb5kdc[2995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:18:23.961230 3026 catalog_manager.cc:4841] TS 081f48bae01247d4b49ae490be5b58d6 (127.15.11.131:33861): tablet 8b2d02be8c3c463883579b66538821ae (table test-table [id=082be13d41f44903948c33fba865ccdf]) successfully deleted
I20260809 15:18:23.961906 3571 ts_tablet_manager.cc:1905] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:23.968524 3540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:23.931064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42758 (local address 127.15.11.129:44381)
0809 15:18:23.931284 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:23.931293 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:23.948614 (+ 17321us) server_negotiation.cc:407] Connection header received
0809 15:18:23.949086 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:23.949091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:23.949167 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:23.949251 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:23.951144 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.952325 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.954134 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:23.954907 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:23.955030 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:23.960736 (+ 5706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:23.960751 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:23.960769 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:23.960810 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:23.964913 (+ 4103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:23.965741 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:23.965752 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:23.965758 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:23.965939 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:23.966453 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:23.966458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:23.966462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:23.966949 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:23.967227 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:23.967854 (+ 627us) server_negotiation.cc:299] Negotiation successful
0809 15:18:23.968099 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 15:18:23.970144 3237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:24.060830 3571 ts_tablet_manager.cc:1918] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:24.061167 3571 log.cc:1196] T 8b2d02be8c3c463883579b66538821ae P c7bbb5a7fb334092a4117b1b553676ad: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/8b2d02be8c3c463883579b66538821ae
I20260809 15:18:24.061542 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 3108
W20260809 15:18:24.084622 3026 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.11.129:44381 (error 108)
I20260809 15:18:24.084970 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 3248
I20260809 15:18:24.086962 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.086407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.0.0.1:42760)
0809 15:18:24.086622 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:24.086748 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 15:18:24.088457 3026 catalog_manager.cc:4584] TS c7bbb5a7fb334092a4117b1b553676ad (127.15.11.129:44381): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b2d02be8c3c463883579b66538821ae: Network error: Client connection negotiation failed: client connection to 127.15.11.129:44381: connect: Connection refused (error 111)
Aug 09 15:18:24 dist-test-slave-8w9v krb5kdc[2995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:18:24.102905 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.086039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44198 (local address 127.15.11.130:43147)
0809 15:18:24.086238 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:24.086244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:24.086266 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:18:24.086337 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:24.086342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:24.086443 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:24.086555 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:24.088257 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:24.089416 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:24.090716 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:24.091349 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:24.091489 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:24.095681 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:24.095698 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:24.095717 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:24.095762 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:24.099814 (+ 4052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:24.100654 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:24.100665 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:24.100669 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:24.100821 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:24.101221 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:24.101227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:24.101230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:24.101728 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:24.102018 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:24.102289 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 15:18:24.102528 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 15:18:24.103550 3376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:24.142277 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.141716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.0.0.1:42776)
0809 15:18:24.141973 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:24.142092 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 15:18:24.143582 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.143100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.0.0.1:42792)
0809 15:18:24.143301 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:24.143393 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:18:24.194669 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 3248
I20260809 15:18:24.204523 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.203918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.0.0.1:42800)
0809 15:18:24.204194 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:24.204317 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 15:18:24.205826 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.205328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.0.0.1:42816)
0809 15:18:24.205523 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:24.205628 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:18:24.218832 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 3387
Aug 09 15:18:24 dist-test-slave-8w9v krb5kdc[2995](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:18:24.235571 3544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.219924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53526 (local address 127.15.11.131:33861)
0809 15:18:24.220178 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:24.220184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:24.220202 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:18:24.220267 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:24.220272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:24.220348 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:24.220460 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:24.221840 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:24.222666 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:24.223942 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:24.224646 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:24.224803 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:24.229220 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:24.229232 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:24.229248 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:24.229289 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:24.232617 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:24.233345 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:24.233364 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:24.233366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:24.233533 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:24.233936 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:24.233943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:24.233951 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:24.234317 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:24.234660 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:24.234941 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 15:18:24.235163 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 15:18:24.236316 3519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:24.287096 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.286519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.0.0.1:42820)
0809 15:18:24.286753 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:24.286938 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 15:18:24.288296 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.287792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.0.0.1:42832)
0809 15:18:24.288051 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:24.288168 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 15:18:24.362094 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 3387
I20260809 15:18:24.386998 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 3009
I20260809 15:18:24.388458 3072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:24.502281 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:24.501435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:44381 (local address 127.0.0.1:42842)
0809 15:18:24.501696 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:24.501819 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:44381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 15:18:24.502233 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 3009
2026-08-09T15:18:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4442 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T15:18:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:24Z Disabled control of system clock
I20260809 15:18:24.591069 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:45625
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:45625
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:18:24.927204 3582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:24.927486 3582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:24.927567 3582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:24.934543 3582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:24.934645 3582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:24.934684 3582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:24.934742 3582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:24.934787 3582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:24.943459 3582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:45625
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:45625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:24.944631 3582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:24.946341 3582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:24.957784 3588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:24.957870 3587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:24.957882 3590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:24.961506 3582 server_base.cc:1034] running on GCE node
I20260809 15:18:24.962227 3582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:24.963544 3582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:24.964776 3582 hybrid_clock.cc:648] HybridClock initialized: now 1786288704964733 us; error 55 us; skew 500 ppm
I20260809 15:18:24.965116 3582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:24.967289 3582 webserver.cc:460] Webserver started at http://127.15.11.190:37655/ using document root <none> and password file <none>
I20260809 15:18:24.967964 3582 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:24.968091 3582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:24.968462 3582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:24.971321 3582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "b28860ce4cd14722afe20f9da5b81007"
format_stamp: "Formatted at 2026-08-09 15:18:24 on dist-test-slave-8w9v"
server_key: "fed472f2a9cc423f492bbbbbcf550958"
server_key_iv: "ca935679752439056e208ece0967208c"
server_key_version: "encryptionkey@0"
I20260809 15:18:24.972179 3582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "b28860ce4cd14722afe20f9da5b81007"
format_stamp: "Formatted at 2026-08-09 15:18:24 on dist-test-slave-8w9v"
server_key: "fed472f2a9cc423f492bbbbbcf550958"
server_key_iv: "ca935679752439056e208ece0967208c"
server_key_version: "encryptionkey@0"
I20260809 15:18:24.977258 3582 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 15:18:24.980600 3596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:24.981909 3582 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 15:18:24.982097 3582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "b28860ce4cd14722afe20f9da5b81007"
format_stamp: "Formatted at 2026-08-09 15:18:24 on dist-test-slave-8w9v"
server_key: "fed472f2a9cc423f492bbbbbcf550958"
server_key_iv: "ca935679752439056e208ece0967208c"
server_key_version: "encryptionkey@0"
I20260809 15:18:24.982306 3582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:24.998155 3582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:24.998901 3582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:24.999171 3582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:25.019308 3582 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:45625
I20260809 15:18:25.019343 3649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:45625 every 8 connection(s)
I20260809 15:18:25.020854 3582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:25.024614 3650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:25.026120 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 3582
I20260809 15:18:25.026340 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:25.026876 15406 external_mini_cluster.cc:1428] Setting key d4fe58d883e6681563019191e57f2372
I20260809 15:18:25.036244 3650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007: Bootstrap starting.
I20260809 15:18:25.039597 3650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:25.040808 3650 log.cc:824] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:25.040845 3653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:25.028919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.15.11.190:45625)
0809 15:18:25.030315 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:25.030326 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:25.030368 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:18:25.032039 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:25.032061 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:25.032578 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:25.032973 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:25.033624 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:25.034887 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:25.036316 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:25.037039 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:25.037233 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:25.038456 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:25.038464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:25.038477 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:25.038984 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:25.039069 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:25.039090 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:25.039211 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:25.039448 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 15:18:25.039792 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":736,"threads_started":1}
I20260809 15:18:25.043840 3650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007: No bootstrap required, opened a new log
I20260809 15:18:25.051261 3650 raft_consensus.cc:348] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28860ce4cd14722afe20f9da5b81007" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45625 } }
I20260809 15:18:25.051591 3650 raft_consensus.cc:374] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:25.051694 3650 raft_consensus.cc:729] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b28860ce4cd14722afe20f9da5b81007, State: Initialized, Role: FOLLOWER
I20260809 15:18:25.052279 3650 consensus_queue.cc:260] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28860ce4cd14722afe20f9da5b81007" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45625 } }
I20260809 15:18:25.052516 3650 raft_consensus.cc:388] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:25.052656 3650 raft_consensus.cc:482] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:25.052827 3650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:25.055251 3650 raft_consensus.cc:504] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28860ce4cd14722afe20f9da5b81007" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45625 } }
I20260809 15:18:25.055757 3650 leader_election.cc:302] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b28860ce4cd14722afe20f9da5b81007; no voters:
I20260809 15:18:25.056375 3650 leader_election.cc:288] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:25.056569 3655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:25.057308 3655 raft_consensus.cc:686] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [term 1 LEADER]: Becoming Leader. State: Replica: b28860ce4cd14722afe20f9da5b81007, State: Running, Role: LEADER
I20260809 15:18:25.057833 3655 consensus_queue.cc:237] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28860ce4cd14722afe20f9da5b81007" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45625 } }
I20260809 15:18:25.058521 3650 sys_catalog.cc:564] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:25.060636 3657 sys_catalog.cc:455] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b28860ce4cd14722afe20f9da5b81007. Latest consensus state: current_term: 1 leader_uuid: "b28860ce4cd14722afe20f9da5b81007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28860ce4cd14722afe20f9da5b81007" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45625 } } }
I20260809 15:18:25.060662 3656 sys_catalog.cc:455] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b28860ce4cd14722afe20f9da5b81007" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b28860ce4cd14722afe20f9da5b81007" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45625 } } }
I20260809 15:18:25.061136 3657 sys_catalog.cc:458] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:25.061237 3656 sys_catalog.cc:458] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:25.063057 3660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:25.068178 3660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:25.076550 3660 catalog_manager.cc:1347] Generated new cluster ID: d9c4e1d1e53e45fbb5b17ad23459da46
I20260809 15:18:25.076668 3660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:25.095243 3660 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:25.096635 3660 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:25.104425 3660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007: Generated new TSK 0
I20260809 15:18:25.105253 3660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:25.114724 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45625
--builtin_ntp_servers=127.15.11.148:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:18:25.455070 3675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:25.455320 3675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:25.455408 3675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:25.463680 3675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:25.463769 3675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:25.463953 3675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:18:25.473038 3675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:25.474246 3675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:25.476064 3675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:25.489013 3680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:25.488969 3683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:25.489248 3681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:25.490197 3675 server_base.cc:1034] running on GCE node
I20260809 15:18:25.490844 3675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:25.491758 3675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:25.493018 3675 hybrid_clock.cc:648] HybridClock initialized: now 1786288705492973 us; error 31 us; skew 500 ppm
I20260809 15:18:25.493350 3675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:25.495616 3675 webserver.cc:460] Webserver started at http://127.15.11.129:43475/ using document root <none> and password file <none>
I20260809 15:18:25.496410 3675 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:25.496546 3675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:25.496873 3675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:25.499620 3675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "0fada3e1eb274d59897ec2b9196f9a46"
format_stamp: "Formatted at 2026-08-09 15:18:25 on dist-test-slave-8w9v"
server_key: "efa61b94194c9ee343b2501726154f76"
server_key_iv: "7e21ef3b862df4f5290bc01ac593c7f9"
server_key_version: "encryptionkey@0"
I20260809 15:18:25.500486 3675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "0fada3e1eb274d59897ec2b9196f9a46"
format_stamp: "Formatted at 2026-08-09 15:18:25 on dist-test-slave-8w9v"
server_key: "efa61b94194c9ee343b2501726154f76"
server_key_iv: "7e21ef3b862df4f5290bc01ac593c7f9"
server_key_version: "encryptionkey@0"
I20260809 15:18:25.505501 3675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:18:25.508978 3689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:25.510221 3675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 15:18:25.510406 3675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "0fada3e1eb274d59897ec2b9196f9a46"
format_stamp: "Formatted at 2026-08-09 15:18:25 on dist-test-slave-8w9v"
server_key: "efa61b94194c9ee343b2501726154f76"
server_key_iv: "7e21ef3b862df4f5290bc01ac593c7f9"
server_key_version: "encryptionkey@0"
I20260809 15:18:25.510617 3675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:25.526549 3675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:25.527406 3675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:25.527733 3675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:25.528759 3675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:25.530217 3675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:25.530339 3675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:25.530478 3675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:25.530567 3675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:25.563580 3675 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:42239
I20260809 15:18:25.563676 3809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:42239 every 8 connection(s)
I20260809 15:18:25.565078 3675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:18:25.575464 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 3675
I20260809 15:18:25.575651 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:18:25.576218 15406 external_mini_cluster.cc:1428] Setting key c58c31be3366b4c969987a3d0c3f655c
I20260809 15:18:25.577690 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:25.567580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:35435 (local address 127.15.11.190:45625)
0809 15:18:25.568072 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:25.568076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:25.569285 (+ 1209us) server_negotiation.cc:407] Connection header received
0809 15:18:25.571062 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:25.571068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:25.571169 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:25.571290 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:25.572450 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:25.573166 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:25.574722 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:25.575189 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:25.575323 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:25.576826 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:25.576832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:25.576836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:25.576858 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:25.576913 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:25.576924 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:25.577021 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:25.577249 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 15:18:25.577341 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":294,"threads_started":1}
I20260809 15:18:25.578631 3813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:25.568224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45625 (local address 127.15.11.129:35435)
0809 15:18:25.569089 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:25.569154 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:25.570767 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:25.571534 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:25.571542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:25.571625 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:25.572273 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:25.572293 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:25.573322 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:25.573335 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:25.574552 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:25.574565 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:25.575445 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:25.575453 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:25.575766 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:25.576521 (+ 755us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:18:25.576524 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:25.576563 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:18:25.576633 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:25.577023 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:25.577036 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:25.577046 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 15:18:25.577218 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 15:18:25.577594 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":275,"threads_started":1}
I20260809 15:18:25.579010 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45625
--builtin_ntp_servers=127.15.11.148:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:18:25.580173 3810 heartbeater.cc:347] Connected to a master server at 127.15.11.190:45625
I20260809 15:18:25.580493 3810 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:25.581401 3810 heartbeater.cc:511] Master 127.15.11.190:45625 requested a full tablet report, sending...
I20260809 15:18:25.583416 3614 ts_manager.cc:194] Registered new tserver with Master: 0fada3e1eb274d59897ec2b9196f9a46 (127.15.11.129:42239)
I20260809 15:18:25.584928 3614 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.129:35435
W20260809 15:18:25.924633 3814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:25.924904 3814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:25.924996 3814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:25.932253 3814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:25.932356 3814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:25.932468 3814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:18:25.941812 3814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:25.943027 3814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:25.944865 3814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:25.958037 3819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:25.958135 3820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:25.958029 3822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:25.958829 3814 server_base.cc:1034] running on GCE node
I20260809 15:18:25.959401 3814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:25.960417 3814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:25.961622 3814 hybrid_clock.cc:648] HybridClock initialized: now 1786288705961585 us; error 30 us; skew 500 ppm
I20260809 15:18:25.961935 3814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:25.964166 3814 webserver.cc:460] Webserver started at http://127.15.11.130:44447/ using document root <none> and password file <none>
I20260809 15:18:25.964859 3814 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:25.964944 3814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:25.965282 3814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:25.968025 3814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "11839988bb0a48499242116f0e5f1975"
format_stamp: "Formatted at 2026-08-09 15:18:25 on dist-test-slave-8w9v"
server_key: "9d801a84fb9b642b4f681dc491726059"
server_key_iv: "200a79d7ab5137d16b14c7d44ced1e64"
server_key_version: "encryptionkey@0"
I20260809 15:18:25.968933 3814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "11839988bb0a48499242116f0e5f1975"
format_stamp: "Formatted at 2026-08-09 15:18:25 on dist-test-slave-8w9v"
server_key: "9d801a84fb9b642b4f681dc491726059"
server_key_iv: "200a79d7ab5137d16b14c7d44ced1e64"
server_key_version: "encryptionkey@0"
I20260809 15:18:25.973874 3814 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:18:25.977337 3829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:25.978569 3814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 15:18:25.978705 3814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "11839988bb0a48499242116f0e5f1975"
format_stamp: "Formatted at 2026-08-09 15:18:25 on dist-test-slave-8w9v"
server_key: "9d801a84fb9b642b4f681dc491726059"
server_key_iv: "200a79d7ab5137d16b14c7d44ced1e64"
server_key_version: "encryptionkey@0"
I20260809 15:18:25.978888 3814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:26.000450 3814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:26.001189 3814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:26.001451 3814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:26.002437 3814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:26.003989 3814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:26.004076 3814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:26.004206 3814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:26.004273 3814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:26.038935 3814 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:35411
I20260809 15:18:26.039040 3952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:35411 every 8 connection(s)
I20260809 15:18:26.040283 3814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:18:26.049639 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 3814
I20260809 15:18:26.049821 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:18:26.050319 15406 external_mini_cluster.cc:1428] Setting key b7aa30aed1b14e01654237eebb584a73
I20260809 15:18:26.052791 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.042895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:56281 (local address 127.15.11.190:45625)
0809 15:18:26.043121 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.043128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.044576 (+ 1448us) server_negotiation.cc:407] Connection header received
0809 15:18:26.046301 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.046305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.046378 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.046466 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:26.047508 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.048279 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.049916 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.050319 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.050462 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.051922 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:26.051928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:26.051933 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:26.051953 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:26.051993 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:26.052011 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:26.052155 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.052392 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.052490 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 15:18:26.053349 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45625
--builtin_ntp_servers=127.15.11.148:34837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:18:26.053879 3955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.043462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45625 (local address 127.15.11.130:56281)
0809 15:18:26.044424 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:26.044466 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:26.046071 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:26.046666 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:26.046673 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:26.046759 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:26.047320 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.047336 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.048426 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.048430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.049743 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.049759 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.050554 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.050562 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.050783 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.051598 (+ 815us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:18:26.051601 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:26.051640 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:18:26.051717 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:26.052165 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:26.052185 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:26.052192 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 15:18:26.052382 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 15:18:26.052772 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":298,"threads_started":1}
I20260809 15:18:26.055320 3953 heartbeater.cc:347] Connected to a master server at 127.15.11.190:45625
I20260809 15:18:26.055620 3953 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:26.056428 3953 heartbeater.cc:511] Master 127.15.11.190:45625 requested a full tablet report, sending...
I20260809 15:18:26.057895 3614 ts_manager.cc:194] Registered new tserver with Master: 11839988bb0a48499242116f0e5f1975 (127.15.11.130:35411)
I20260809 15:18:26.059163 3614 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.130:56281
W20260809 15:18:26.392100 3957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:26.392400 3957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:26.392549 3957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:26.399473 3957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:26.399561 3957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:26.399716 3957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:18:26.408514 3957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:34837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:45625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:26.409776 3957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:26.411550 3957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:26.424855 3963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:26.425012 3965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:26.425042 3962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:26.425801 3957 server_base.cc:1034] running on GCE node
I20260809 15:18:26.426312 3957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:26.427155 3957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:26.428354 3957 hybrid_clock.cc:648] HybridClock initialized: now 1786288706428312 us; error 31 us; skew 500 ppm
I20260809 15:18:26.428684 3957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:26.430887 3957 webserver.cc:460] Webserver started at http://127.15.11.131:36527/ using document root <none> and password file <none>
I20260809 15:18:26.431564 3957 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:26.431681 3957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:26.432003 3957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:26.434907 3957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "4215819aa36244d68dab60b782c10c07"
format_stamp: "Formatted at 2026-08-09 15:18:26 on dist-test-slave-8w9v"
server_key: "e6c4cbe0c763a8c47cdc1b3d89194c42"
server_key_iv: "fe2c786c175edcd92b44b94d7b375d27"
server_key_version: "encryptionkey@0"
I20260809 15:18:26.435701 3957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "4215819aa36244d68dab60b782c10c07"
format_stamp: "Formatted at 2026-08-09 15:18:26 on dist-test-slave-8w9v"
server_key: "e6c4cbe0c763a8c47cdc1b3d89194c42"
server_key_iv: "fe2c786c175edcd92b44b94d7b375d27"
server_key_version: "encryptionkey@0"
I20260809 15:18:26.440661 3957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:18:26.444077 3971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:26.445423 3957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:18:26.445595 3957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "4215819aa36244d68dab60b782c10c07"
format_stamp: "Formatted at 2026-08-09 15:18:26 on dist-test-slave-8w9v"
server_key: "e6c4cbe0c763a8c47cdc1b3d89194c42"
server_key_iv: "fe2c786c175edcd92b44b94d7b375d27"
server_key_version: "encryptionkey@0"
I20260809 15:18:26.445802 3957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:26.474352 3957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:26.475201 3957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:26.475529 3957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:26.476601 3957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:26.478255 3957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:26.478385 3957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:26.478533 3957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:26.478636 3957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:26.513556 3957 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:40107
I20260809 15:18:26.513661 4089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:40107 every 8 connection(s)
I20260809 15:18:26.515128 3957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:18:26.515484 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 3957
I20260809 15:18:26.515692 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:18:26.516346 15406 external_mini_cluster.cc:1428] Setting key cceee1caed4982ee56f63117a3336668
I20260809 15:18:26.528316 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.518274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:38649 (local address 127.15.11.190:45625)
0809 15:18:26.518481 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.518487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.520322 (+ 1835us) server_negotiation.cc:407] Connection header received
0809 15:18:26.522168 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.522173 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.522266 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.522385 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:26.523454 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.524293 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.525887 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.526310 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.526410 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.527615 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:26.527619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:26.527623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:26.527639 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:26.527670 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:26.527682 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:26.527751 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.527945 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.528024 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:18:26.529450 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.518965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:45625 (local address 127.15.11.131:38649)
0809 15:18:26.520174 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:26.520213 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:26.521902 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:26.522623 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:26.522631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:26.522719 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:26.523280 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.523296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.524479 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.524483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.525741 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.525753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.526516 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.526520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.526714 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.527325 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:18:26.527327 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:26.527370 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:18:26.527441 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:26.527766 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:26.527773 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:26.527778 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:18:26.527925 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 15:18:26.528259 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":403,"threads_started":1}
I20260809 15:18:26.531029 4090 heartbeater.cc:347] Connected to a master server at 127.15.11.190:45625
I20260809 15:18:26.531352 4090 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:26.532260 4090 heartbeater.cc:511] Master 127.15.11.190:45625 requested a full tablet report, sending...
I20260809 15:18:26.533635 3614 ts_manager.cc:194] Registered new tserver with Master: 4215819aa36244d68dab60b782c10c07 (127.15.11.131:40107)
I20260809 15:18:26.534386 3614 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.131:38649
I20260809 15:18:26.544395 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:18:26.554819 3812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.548568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60794 (local address 127.15.11.190:45625)
0809 15:18:26.548727 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.548731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.549279 (+ 548us) server_negotiation.cc:407] Connection header received
0809 15:18:26.549496 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.549501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.549562 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.549661 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:26.550211 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.550906 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.552272 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.552688 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.552824 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.554020 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:26.554025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:26.554028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:26.554044 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:26.554076 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:26.554087 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:26.554259 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.554489 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.554547 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:18:26.558588 3614 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:18:26.566816 3614 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:18:26.588467 3810 heartbeater.cc:503] Master 127.15.11.190:45625 was elected leader, sending a full tablet report...
I20260809 15:18:26.592109 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.578983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:40107 (local address 127.0.0.1:35046)
0809 15:18:26.580674 (+ 1691us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:26.580733 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:26.580945 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:26.582671 (+ 1726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:26.582679 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:26.582709 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:26.583160 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.583176 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.585541 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.585547 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.588206 (+ 2659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.588218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.590583 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.590592 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.590832 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.591310 (+ 478us) client_negotiation.cc:769] Sending connection context
0809 15:18:26.591438 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 15:18:26.591650 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1396,"mutex_wait_us":428,"thread_start_us":3087,"threads_started":1}
I20260809 15:18:26.592375 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.579924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53562 (local address 127.15.11.130:35411)
0809 15:18:26.582108 (+ 2184us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.582116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.582633 (+ 517us) server_negotiation.cc:407] Connection header received
0809 15:18:26.582728 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.582736 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.582981 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.583241 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:26.584183 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.585680 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.588844 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.590323 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.590973 (+ 650us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.591223 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.591505 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.591964 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1987,"thread_start_us":1858,"threads_started":1}
I20260809 15:18:26.592741 4105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.579245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.15.11.131:40107)
0809 15:18:26.581703 (+ 2458us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.581715 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.581757 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:18:26.581866 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.581873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.582317 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.583336 (+ 1019us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:26.583659 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.585383 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.588924 (+ 3541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.590323 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.590961 (+ 638us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.591701 (+ 740us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.591802 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.592003 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2212,"thread_start_us":2055,"threads_started":1}
I20260809 15:18:26.593130 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.579143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:35411 (local address 127.0.0.1:53562)
0809 15:18:26.582373 (+ 3230us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:26.582398 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:26.582520 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:26.583514 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:26.583519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:26.583540 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:26.583928 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.583944 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.585834 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.585844 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.588619 (+ 2775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.588632 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.590635 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.590642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.591241 (+ 599us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.591307 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 15:18:26.591425 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 15:18:26.592752 (+ 1327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3089,"mutex_wait_us":312,"thread_start_us":1068,"threads_started":1}
I20260809 15:18:26.596237 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.583502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:42239 (local address 127.0.0.1:34710)
0809 15:18:26.587889 (+ 4387us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:26.587906 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:26.588015 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:26.589285 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:26.589289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:26.589313 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:26.589761 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.589773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.591993 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.592001 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.594204 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.594217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.595488 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.595494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.595656 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.595697 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 15:18:26.595788 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:18:26.595905 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4271,"thread_start_us":4132,"threads_started":1}
I20260809 15:18:26.596649 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.582794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.15.11.129:42239)
0809 15:18:26.588794 (+ 6000us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.588803 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.588830 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:18:26.588919 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.588927 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.589108 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.589301 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:26.589958 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.591817 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.594382 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.595279 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.595429 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.595706 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.596034 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.596263 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5825,"thread_start_us":5700,"threads_started":1}
I20260809 15:18:26.596872 3884 tablet_service.cc:1467] Processing CreateTablet for tablet 9345927a09334428871e12ee42fef3a4 (DEFAULT_TABLE table=test-table [id=efb413fd8979486da89e227128f2acf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:26.596861 4023 tablet_service.cc:1467] Processing CreateTablet for tablet 9345927a09334428871e12ee42fef3a4 (DEFAULT_TABLE table=test-table [id=efb413fd8979486da89e227128f2acf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:26.598477 4023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9345927a09334428871e12ee42fef3a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:26.599207 3884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9345927a09334428871e12ee42fef3a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:26.600364 3738 tablet_service.cc:1467] Processing CreateTablet for tablet 9345927a09334428871e12ee42fef3a4 (DEFAULT_TABLE table=test-table [id=efb413fd8979486da89e227128f2acf7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:26.601769 3738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9345927a09334428871e12ee42fef3a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:26.611174 4109 tablet_bootstrap.cc:492] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: Bootstrap starting.
I20260809 15:18:26.613865 4110 tablet_bootstrap.cc:492] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: Bootstrap starting.
I20260809 15:18:26.614868 4109 tablet_bootstrap.cc:654] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:26.616014 4111 tablet_bootstrap.cc:492] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46: Bootstrap starting.
I20260809 15:18:26.616225 4109 log.cc:824] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:26.617404 4110 tablet_bootstrap.cc:654] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:26.619310 4111 tablet_bootstrap.cc:654] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:26.618659 4110 log.cc:824] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:26.619639 4109 tablet_bootstrap.cc:492] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: No bootstrap required, opened a new log
I20260809 15:18:26.619982 4109 ts_tablet_manager.cc:1397] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:18:26.620739 4111 log.cc:824] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:26.623186 4110 tablet_bootstrap.cc:492] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: No bootstrap required, opened a new log
I20260809 15:18:26.623533 4110 ts_tablet_manager.cc:1397] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20260809 15:18:26.625803 4111 tablet_bootstrap.cc:492] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46: No bootstrap required, opened a new log
I20260809 15:18:26.626075 4111 ts_tablet_manager.cc:1397] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 15:18:26.627590 4109 raft_consensus.cc:348] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } }
I20260809 15:18:26.627954 4109 raft_consensus.cc:374] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:26.628080 4109 raft_consensus.cc:729] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11839988bb0a48499242116f0e5f1975, State: Initialized, Role: FOLLOWER
I20260809 15:18:26.628753 4109 consensus_queue.cc:260] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } }
I20260809 15:18:26.629916 3953 heartbeater.cc:503] Master 127.15.11.190:45625 was elected leader, sending a full tablet report...
I20260809 15:18:26.631922 4109 ts_tablet_manager.cc:1428] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: Time spent starting tablet: real 0.012s user 0.006s sys 0.004s
I20260809 15:18:26.634320 4110 raft_consensus.cc:348] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } }
I20260809 15:18:26.634771 4110 raft_consensus.cc:374] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:26.634879 4110 raft_consensus.cc:729] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4215819aa36244d68dab60b782c10c07, State: Initialized, Role: FOLLOWER
I20260809 15:18:26.635504 4110 consensus_queue.cc:260] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } }
I20260809 15:18:26.636629 4090 heartbeater.cc:503] Master 127.15.11.190:45625 was elected leader, sending a full tablet report...
I20260809 15:18:26.636708 4111 raft_consensus.cc:348] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } }
I20260809 15:18:26.637164 4111 raft_consensus.cc:374] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:26.637297 4111 raft_consensus.cc:729] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fada3e1eb274d59897ec2b9196f9a46, State: Initialized, Role: FOLLOWER
I20260809 15:18:26.637466 4110 ts_tablet_manager.cc:1428] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260809 15:18:26.637943 4111 consensus_queue.cc:260] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } }
I20260809 15:18:26.639432 4111 ts_tablet_manager.cc:1428] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260809 15:18:26.767061 4091 tablet.cc:2369] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:26.788301 4117 raft_consensus.cc:482] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:18:26.788581 4117 raft_consensus.cc:504] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } }
I20260809 15:18:26.790340 4117 leader_election.cc:288] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11839988bb0a48499242116f0e5f1975 (127.15.11.130:35411), 0fada3e1eb274d59897ec2b9196f9a46 (127.15.11.129:42239)
W20260809 15:18:26.792335 3954 tablet.cc:2369] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:26.797044 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.790986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:48645 (local address 127.15.11.129:42239)
0809 15:18:26.791206 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.791213 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.791319 (+ 106us) server_negotiation.cc:407] Connection header received
0809 15:18:26.791529 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.791537 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.791612 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.791708 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:26.792207 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.793105 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.795245 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.796065 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.796285 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.796359 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.796629 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.796764 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 15:18:26.797050 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.791014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:42239 (local address 127.15.11.131:48645)
0809 15:18:26.791198 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:26.791220 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:26.791373 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:26.791743 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:26.791749 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:26.791780 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:26.791972 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.791980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.793248 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.793256 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.795073 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.795088 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.796273 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.796279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.796451 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.796474 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:18:26.796573 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 15:18:26.796725 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 15:18:26.797812 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.790743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:33395 (local address 127.15.11.130:35411)
0809 15:18:26.790987 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.790995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.792080 (+ 1085us) server_negotiation.cc:407] Connection header received
0809 15:18:26.792269 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.792294 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.792414 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.792520 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:26.793346 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.794385 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.796065 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.796898 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.797075 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.797135 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.797455 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.797577 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:18:26.797838 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.791048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:35411 (local address 127.15.11.131:33395)
0809 15:18:26.791924 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:26.791937 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:26.792109 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:26.792648 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:26.792652 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:26.792679 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:26.793098 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.793116 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.794525 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.794529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.795872 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:26.795883 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.797083 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.797089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:26.797263 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.797294 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 15:18:26.797427 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 15:18:26.797542 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":535,"threads_started":1}
I20260809 15:18:26.798142 3761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9345927a09334428871e12ee42fef3a4" candidate_uuid: "4215819aa36244d68dab60b782c10c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fada3e1eb274d59897ec2b9196f9a46" is_pre_election: true
I20260809 15:18:26.798725 3761 raft_consensus.cc:2454] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4215819aa36244d68dab60b782c10c07 in term 0.
I20260809 15:18:26.798840 3906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9345927a09334428871e12ee42fef3a4" candidate_uuid: "4215819aa36244d68dab60b782c10c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11839988bb0a48499242116f0e5f1975" is_pre_election: true
I20260809 15:18:26.799377 3906 raft_consensus.cc:2454] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4215819aa36244d68dab60b782c10c07 in term 0.
I20260809 15:18:26.799412 3974 leader_election.cc:302] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fada3e1eb274d59897ec2b9196f9a46, 4215819aa36244d68dab60b782c10c07; no voters:
I20260809 15:18:26.799795 4117 raft_consensus.cc:2781] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:18:26.800002 4117 raft_consensus.cc:482] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:18:26.800144 4117 raft_consensus.cc:3037] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:26.802354 4117 raft_consensus.cc:504] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } }
I20260809 15:18:26.802942 4117 leader_election.cc:288] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [CANDIDATE]: Term 1 election: Requested vote from peers 11839988bb0a48499242116f0e5f1975 (127.15.11.130:35411), 0fada3e1eb274d59897ec2b9196f9a46 (127.15.11.129:42239)
I20260809 15:18:26.803249 3906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9345927a09334428871e12ee42fef3a4" candidate_uuid: "4215819aa36244d68dab60b782c10c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11839988bb0a48499242116f0e5f1975"
I20260809 15:18:26.803452 3906 raft_consensus.cc:3037] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:26.803414 3761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9345927a09334428871e12ee42fef3a4" candidate_uuid: "4215819aa36244d68dab60b782c10c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fada3e1eb274d59897ec2b9196f9a46"
I20260809 15:18:26.803624 3761 raft_consensus.cc:3037] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:26.805896 3761 raft_consensus.cc:2454] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4215819aa36244d68dab60b782c10c07 in term 1.
I20260809 15:18:26.805932 3906 raft_consensus.cc:2454] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4215819aa36244d68dab60b782c10c07 in term 1.
I20260809 15:18:26.806305 3974 leader_election.cc:302] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fada3e1eb274d59897ec2b9196f9a46, 4215819aa36244d68dab60b782c10c07; no voters:
I20260809 15:18:26.806643 4117 raft_consensus.cc:2781] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:26.807113 4117 raft_consensus.cc:686] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 1 LEADER]: Becoming Leader. State: Replica: 4215819aa36244d68dab60b782c10c07, State: Running, Role: LEADER
I20260809 15:18:26.807554 4117 consensus_queue.cc:237] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } }
I20260809 15:18:26.812228 3613 catalog_manager.cc:5495] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4215819aa36244d68dab60b782c10c07 (127.15.11.131). New cstate: current_term: 1 leader_uuid: "4215819aa36244d68dab60b782c10c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4215819aa36244d68dab60b782c10c07" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 40107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 } health_report { overall_health: UNKNOWN } } }
W20260809 15:18:26.817471 3811 tablet.cc:2369] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:26.828877 4105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.823745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.15.11.131:40107)
0809 15:18:26.823926 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.823930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.823947 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:18:26.824162 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.824166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.824239 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.824355 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:18:26.824748 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.825441 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.826777 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.827207 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.827296 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.827771 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:18:26.828370 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:18:26.828516 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.828650 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.828684 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:18:26.836669 3906 raft_consensus.cc:1261] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [term 1 FOLLOWER]: Refusing update from remote peer 4215819aa36244d68dab60b782c10c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:26.836675 3761 raft_consensus.cc:1261] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [term 1 FOLLOWER]: Refusing update from remote peer 4215819aa36244d68dab60b782c10c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:26.837641 4120 consensus_queue.cc:1035] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fada3e1eb274d59897ec2b9196f9a46" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:26.838025 4117 consensus_queue.cc:1035] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11839988bb0a48499242116f0e5f1975" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 35411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:26.862957 3613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:26.863240 3613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:26.866632 3613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b28860ce4cd14722afe20f9da5b81007: Sending DeleteTablet for 3 replicas of tablet 9345927a09334428871e12ee42fef3a4
I20260809 15:18:26.867733 4023 tablet_service.cc:1514] Processing DeleteTablet for tablet 9345927a09334428871e12ee42fef3a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:26 UTC) from {username='slave'} at 127.0.0.1:35046
I20260809 15:18:26.867830 3884 tablet_service.cc:1514] Processing DeleteTablet for tablet 9345927a09334428871e12ee42fef3a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:26 UTC) from {username='slave'} at 127.0.0.1:53562
I20260809 15:18:26.868716 4136 tablet_replica.cc:330] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: stopping tablet replica
I20260809 15:18:26.868880 3738 tablet_service.cc:1514] Processing DeleteTablet for tablet 9345927a09334428871e12ee42fef3a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:26 UTC) from {username='slave'} at 127.0.0.1:34710
I20260809 15:18:26.869185 4136 raft_consensus.cc:2229] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:26.869714 4136 raft_consensus.cc:2258] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:26.869762 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 3675
I20260809 15:18:26.870404 4135 tablet_replica.cc:330] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: stopping tablet replica
I20260809 15:18:26.870855 4135 raft_consensus.cc:2229] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:18:26.871599 4135 raft_consensus.cc:2258] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:26.874156 4135 ts_tablet_manager.cc:1905] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:26.874856 4136 ts_tablet_manager.cc:1905] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:26.881371 4107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:26.870842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34716 (local address 127.15.11.129:42239)
0809 15:18:26.871016 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:26.871022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:26.873191 (+ 2169us) server_negotiation.cc:407] Connection header received
0809 15:18:26.873385 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:26.873400 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:26.873505 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:26.873601 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:26.874214 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.875364 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.877320 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:26.877963 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:26.878064 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:26.879600 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:26.879612 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:26.879627 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:26.880412 (+ 785us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:26.880471 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:26.880497 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:26.880605 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:26.880857 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 15:18:26.880974 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:18:26.882752 4137 tablet_replica.cc:330] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46: stopping tablet replica
I20260809 15:18:26.882949 3804 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:26.883474 4135 ts_tablet_manager.cc:1918] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:26.883661 4135 log.cc:1196] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/9345927a09334428871e12ee42fef3a4
I20260809 15:18:26.884241 4135 ts_tablet_manager.cc:1939] T 9345927a09334428871e12ee42fef3a4 P 4215819aa36244d68dab60b782c10c07: Deleting consensus metadata
I20260809 15:18:26.885304 4136 ts_tablet_manager.cc:1918] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:26.885496 4136 log.cc:1196] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/9345927a09334428871e12ee42fef3a4
I20260809 15:18:26.886150 4136 ts_tablet_manager.cc:1939] T 9345927a09334428871e12ee42fef3a4 P 11839988bb0a48499242116f0e5f1975: Deleting consensus metadata
I20260809 15:18:26.886595 3599 catalog_manager.cc:4841] TS 4215819aa36244d68dab60b782c10c07 (127.15.11.131:40107): tablet 9345927a09334428871e12ee42fef3a4 (table test-table [id=efb413fd8979486da89e227128f2acf7]) successfully deleted
I20260809 15:18:26.888334 3598 catalog_manager.cc:4841] TS 11839988bb0a48499242116f0e5f1975 (127.15.11.130:35411): tablet 9345927a09334428871e12ee42fef3a4 (table test-table [id=efb413fd8979486da89e227128f2acf7]) successfully deleted
I20260809 15:18:26.883260 4137 raft_consensus.cc:2229] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:26.975507 4137 raft_consensus.cc:2258] T 9345927a09334428871e12ee42fef3a4 P 0fada3e1eb274d59897ec2b9196f9a46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:26.976677 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 3675
W20260809 15:18:27.000372 3599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.11.129:42239 (error 108)
I20260809 15:18:27.000509 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 3814
I20260809 15:18:27.002700 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:27.002140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:42239 (local address 127.0.0.1:34720)
0809 15:18:27.002368 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:27.002510 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 15:18:27.003010 3599 catalog_manager.cc:4584] TS 0fada3e1eb274d59897ec2b9196f9a46 (127.15.11.129:42239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9345927a09334428871e12ee42fef3a4: Network error: Client connection negotiation failed: client connection to 127.15.11.129:42239: connect: Connection refused (error 111)
I20260809 15:18:27.009258 4106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:27.001774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53568 (local address 127.15.11.130:35411)
0809 15:18:27.001954 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:27.001960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:27.001980 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:18:27.002050 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:27.002056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:27.002171 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:27.002276 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:27.002979 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:27.003907 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:27.005878 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:27.006355 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:27.006466 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:27.007669 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:27.007679 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:27.007694 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:27.008406 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:27.008458 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:27.008481 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:27.008580 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:27.008750 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 15:18:27.008846 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:18:27.010031 3948 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:27.056741 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:27.056213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:42239 (local address 127.0.0.1:34726)
0809 15:18:27.056438 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:27.056549 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 15:18:27.057983 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:27.057572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:42239 (local address 127.0.0.1:34730)
0809 15:18:27.057738 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:27.057830 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:18:27.102524 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 3814
I20260809 15:18:27.118920 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:27.118353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:42239 (local address 127.0.0.1:34740)
0809 15:18:27.118606 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:27.118724 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 15:18:27.120251 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:27.119737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:42239 (local address 127.0.0.1:34756)
0809 15:18:27.119931 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:27.120041 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 15:18:27.124270 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 3957
I20260809 15:18:27.132246 4105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:27.125133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35064 (local address 127.15.11.131:40107)
0809 15:18:27.125302 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:27.125307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:27.125328 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:18:27.125571 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:27.125575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:27.125705 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:27.125814 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:27.126334 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:27.127160 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:27.129062 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:27.129494 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:27.129583 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:27.130785 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:27.130793 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:27.130807 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:27.131417 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:27.131460 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:27.131479 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:27.131557 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:27.131742 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:18:27.131833 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:18:27.133013 4085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:27.201117 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:27.200578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:42239 (local address 127.0.0.1:34770)
0809 15:18:27.200811 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:27.200937 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 15:18:27.202209 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:27.201833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:42239 (local address 127.0.0.1:34782)
0809 15:18:27.202035 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:27.202104 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260809 15:18:27.255282 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 3957
I20260809 15:18:27.280527 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 3582
I20260809 15:18:27.281629 3644 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:27.398293 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 3582
2026-08-09T15:18:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2895 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:27 dist-test-slave-8w9v krb5kdc[4147](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:27 dist-test-slave-8w9v krb5kdc[4147](info): set up 2 sockets
Aug 09 15:18:27 dist-test-slave-8w9v krb5kdc[4147](info): commencing operation
krb5kdc: starting...
W20260809 15:18:28.556411 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:28 dist-test-slave-8w9v krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:18:28.837345 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:33175
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:33175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:29.182130 4160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:29.182421 4160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:29.182528 4160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:29.189713 4160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:29.189826 4160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:29.189893 4160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:29.189975 4160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:29.190018 4160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:29.199522 4160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:33175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:33175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:29.200829 4160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:29.202143 4160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:29.214185 4168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:29.214298 4167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:29.214275 4170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:29.215257 4160 server_base.cc:1034] running on GCE node
I20260809 15:18:29.216032 4160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:29.217401 4160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:29.218597 4160 hybrid_clock.cc:648] HybridClock initialized: now 1786288709218554 us; error 35 us; skew 500 ppm
I20260809 15:18:29.218931 4160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:29 dist-test-slave-8w9v krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288709, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:29.227224 4160 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:18:29.228732 4160 webserver.cc:460] Webserver started at http://127.15.11.190:41943/ using document root <none> and password file <none>
I20260809 15:18:29.229398 4160 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:29.229537 4160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:29.229833 4160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:29.232738 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9"
format_stamp: "Formatted at 2026-08-09 15:18:29 on dist-test-slave-8w9v"
server_key: "43d964b0911513de9b462538b49ac9c4"
server_key_iv: "20dd58e9bb1d3b245f3c561995b39e68"
server_key_version: "encryptionkey@0"
I20260809 15:18:29.233474 4160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9"
format_stamp: "Formatted at 2026-08-09 15:18:29 on dist-test-slave-8w9v"
server_key: "43d964b0911513de9b462538b49ac9c4"
server_key_iv: "20dd58e9bb1d3b245f3c561995b39e68"
server_key_version: "encryptionkey@0"
I20260809 15:18:29.238482 4160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 15:18:29.241827 4177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:29.243327 4160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:18:29.243537 4160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9"
format_stamp: "Formatted at 2026-08-09 15:18:29 on dist-test-slave-8w9v"
server_key: "43d964b0911513de9b462538b49ac9c4"
server_key_iv: "20dd58e9bb1d3b245f3c561995b39e68"
server_key_version: "encryptionkey@0"
I20260809 15:18:29.243732 4160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:29.268975 4160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:29.272702 4160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:29.273000 4160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:29.290547 4160 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:33175
I20260809 15:18:29.290576 4230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:33175 every 8 connection(s)
I20260809 15:18:29.291850 4160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:29.293434 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 4160
I20260809 15:18:29.293638 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:29.294262 15406 external_mini_cluster.cc:1428] Setting key 69f34e9abb3f39f4b16c0f129eb0e3ee
I20260809 15:18:29.296633 4231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:18:29 dist-test-slave-8w9v krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:29.309996 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9: Bootstrap starting.
I20260809 15:18:29.314169 4231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:29.315408 4231 log.cc:824] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:29.318202 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:29.296198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.15.11.190:33175)
0809 15:18:29.297031 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:29.297042 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:29.297081 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:18:29.298718 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:29.298739 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:29.299108 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:29.299434 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:29.301376 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:29.302696 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:29.304541 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:29.305179 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:29.305298 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:29.310056 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:29.310068 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:29.310087 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:29.310133 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:29.314026 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:29.314731 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:29.314739 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:29.314746 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:29.314967 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:29.315316 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:29.315321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:29.315324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:29.316070 (+ 746us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:29.316433 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:29.316880 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 15:18:29.317298 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":363,"threads_started":1}
I20260809 15:18:29.318342 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9: No bootstrap required, opened a new log
I20260809 15:18:29.325774 4231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33175 } }
I20260809 15:18:29.326159 4231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:29.326293 4231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f903ee9c2bf4ea78c933d68d38cb6f9, State: Initialized, Role: FOLLOWER
I20260809 15:18:29.326921 4231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33175 } }
I20260809 15:18:29.327152 4231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:29.327282 4231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:29.327490 4231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:29.329703 4231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33175 } }
I20260809 15:18:29.330209 4231 leader_election.cc:302] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f903ee9c2bf4ea78c933d68d38cb6f9; no voters:
I20260809 15:18:29.330818 4231 leader_election.cc:288] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:29.330957 4237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:29.331677 4237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [term 1 LEADER]: Becoming Leader. State: Replica: 6f903ee9c2bf4ea78c933d68d38cb6f9, State: Running, Role: LEADER
I20260809 15:18:29.332352 4237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33175 } }
I20260809 15:18:29.333128 4231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:29.335639 4238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33175 } } }
I20260809 15:18:29.335592 4239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f903ee9c2bf4ea78c933d68d38cb6f9. Latest consensus state: current_term: 1 leader_uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f903ee9c2bf4ea78c933d68d38cb6f9" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33175 } } }
I20260809 15:18:29.336062 4238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:29.336093 4239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:29.338012 4245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:29.343488 4245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:29.352932 4245 catalog_manager.cc:1347] Generated new cluster ID: 404427b32a664ca8b9e568b350e9f7b0
I20260809 15:18:29.353052 4245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:29.370604 4245 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:29.371966 4245 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:29.378805 4245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9: Generated new TSK 0
I20260809 15:18:29.379571 4245 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
Entry for principal kudu/127.15.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab.
I20260809 15:18:29.445019 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33175
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:29.788610 4258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:29.788856 4258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:29.789008 4258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:29.795746 4258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:29.795856 4258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:29.796036 4258 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:18:29.805050 4258 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.129.keytab
--principal=kudu/127.15.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:29.806274 4258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:29.807587 4258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:29.820441 4264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:29.820537 4263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:29.820634 4266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:29.821596 4258 server_base.cc:1034] running on GCE node
I20260809 15:18:29.822152 4258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:29.823004 4258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:29.824210 4258 hybrid_clock.cc:648] HybridClock initialized: now 1786288709824166 us; error 47 us; skew 500 ppm
I20260809 15:18:29.824510 4258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:29 dist-test-slave-8w9v krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288709, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:29.827385 4258 init.cc:377] Logged in from keytab as kudu/127.15.11.129@KRBTEST.COM (short username kudu)
I20260809 15:18:29.828704 4258 webserver.cc:460] Webserver started at http://127.15.11.129:41041/ using document root <none> and password file <none>
I20260809 15:18:29.829301 4258 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:29.829443 4258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:29.829723 4258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:29.832494 4258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "a906ccc97fb44e408ad00b4b015ea2df"
format_stamp: "Formatted at 2026-08-09 15:18:29 on dist-test-slave-8w9v"
server_key: "e36b346dbec5b88a3fac6699384f847a"
server_key_iv: "bea93ff07499c23d383e36d2be8570da"
server_key_version: "encryptionkey@0"
I20260809 15:18:29.833253 4258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "a906ccc97fb44e408ad00b4b015ea2df"
format_stamp: "Formatted at 2026-08-09 15:18:29 on dist-test-slave-8w9v"
server_key: "e36b346dbec5b88a3fac6699384f847a"
server_key_iv: "bea93ff07499c23d383e36d2be8570da"
server_key_version: "encryptionkey@0"
I20260809 15:18:29.838789 4258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 15:18:29.842422 4273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:29.843722 4258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:18:29.843855 4258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "a906ccc97fb44e408ad00b4b015ea2df"
format_stamp: "Formatted at 2026-08-09 15:18:29 on dist-test-slave-8w9v"
server_key: "e36b346dbec5b88a3fac6699384f847a"
server_key_iv: "bea93ff07499c23d383e36d2be8570da"
server_key_version: "encryptionkey@0"
I20260809 15:18:29.844015 4258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:29.873910 4258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:29.877822 4258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:29.878110 4258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:29.879113 4258 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:29.880486 4258 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:29.880587 4258 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:29.880733 4258 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:29.880810 4258 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:29.912703 4258 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:33295
I20260809 15:18:29.912782 4392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:33295 every 8 connection(s)
I20260809 15:18:29.913961 4258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:18:29.923300 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 4258
I20260809 15:18:29.923492 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:18:29.924054 15406 external_mini_cluster.cc:1428] Setting key c9411e4794ef92a015864cb31265ae50
Aug 09 15:18:29 dist-test-slave-8w9v krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288709, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.129@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:29.935178 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:29.916409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:45701 (local address 127.15.11.190:33175)
0809 15:18:29.916857 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:29.916870 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:29.917637 (+ 767us) server_negotiation.cc:407] Connection header received
0809 15:18:29.919501 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:29.919507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:29.919601 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:29.919714 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:29.921254 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:29.922187 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:29.924426 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:29.925023 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:29.925208 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:29.929388 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:29.929393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:29.929396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:29.929429 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:29.931965 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:29.932711 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:29.932716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:29.932718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:29.932882 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:29.933396 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:29.933401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:29.933404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:29.933670 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:29.933822 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:29.934606 (+ 784us) server_negotiation.cc:299] Negotiation successful
0809 15:18:29.934822 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":253,"threads_started":1}
I20260809 15:18:29.936106 4397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:29.916828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:33175 (local address 127.15.11.129:45701)
0809 15:18:29.917483 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:29.917527 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:29.919236 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:29.919945 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:29.919952 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:29.920536 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:29.921086 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:29.921103 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:29.922394 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:29.922398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:29.924279 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:29.924295 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:29.925343 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:29.925353 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:29.925639 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:29.926378 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:18:29.926428 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:29.929131 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:29.932100 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:29.932108 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:29.932143 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:29.932584 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:29.933006 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:29.933011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:29.933014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:29.933262 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:29.933804 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:29.933813 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:29.934234 (+ 421us) client_negotiation.cc:769] Sending connection context
0809 15:18:29.934495 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 15:18:29.935028 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":274,"threads_started":1}
I20260809 15:18:29.937805 4394 heartbeater.cc:347] Connected to a master server at 127.15.11.190:33175
I20260809 15:18:29.938108 4394 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:29.938977 4394 heartbeater.cc:511] Master 127.15.11.190:33175 requested a full tablet report, sending...
I20260809 15:18:29.941213 4194 ts_manager.cc:194] Registered new tserver with Master: a906ccc97fb44e408ad00b4b015ea2df (127.15.11.129:33295)
I20260809 15:18:29.943341 4194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.129@KRBTEST.COM'} at 127.15.11.129:45701
WARNING: no policy specified for kudu/127.15.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
Entry for principal kudu/127.15.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab.
I20260809 15:18:29.991406 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33175
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:30.325801 4400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:30.326068 4400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:30.326159 4400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:30.333361 4400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:30.333460 4400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:30.333573 4400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:18:30.343007 4400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.130.keytab
--principal=kudu/127.15.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:30.344213 4400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:30.345535 4400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:30.358208 4408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:30.358263 4406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:30.358305 4405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:30.359371 4400 server_base.cc:1034] running on GCE node
I20260809 15:18:30.359977 4400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:30.360920 4400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:30.362146 4400 hybrid_clock.cc:648] HybridClock initialized: now 1786288710362100 us; error 54 us; skew 500 ppm
I20260809 15:18:30.362452 4400 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:30 dist-test-slave-8w9v krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288710, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:30.365631 4400 init.cc:377] Logged in from keytab as kudu/127.15.11.130@KRBTEST.COM (short username kudu)
I20260809 15:18:30.367072 4400 webserver.cc:460] Webserver started at http://127.15.11.130:42897/ using document root <none> and password file <none>
I20260809 15:18:30.367725 4400 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:30.367871 4400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:30.368237 4400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:30.371090 4400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb"
format_stamp: "Formatted at 2026-08-09 15:18:30 on dist-test-slave-8w9v"
server_key: "b59896f62b471aaf961dc6b46486de48"
server_key_iv: "cb498706a51c51e9460a754fdea01c10"
server_key_version: "encryptionkey@0"
I20260809 15:18:30.371872 4400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb"
format_stamp: "Formatted at 2026-08-09 15:18:30 on dist-test-slave-8w9v"
server_key: "b59896f62b471aaf961dc6b46486de48"
server_key_iv: "cb498706a51c51e9460a754fdea01c10"
server_key_version: "encryptionkey@0"
I20260809 15:18:30.377331 4400 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 15:18:30.380579 4415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:30.381837 4400 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:18:30.382010 4400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb"
format_stamp: "Formatted at 2026-08-09 15:18:30 on dist-test-slave-8w9v"
server_key: "b59896f62b471aaf961dc6b46486de48"
server_key_iv: "cb498706a51c51e9460a754fdea01c10"
server_key_version: "encryptionkey@0"
I20260809 15:18:30.382194 4400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:30.397620 4400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:30.402061 4400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:30.402359 4400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:30.403303 4400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:30.404646 4400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:30.404744 4400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:30.404893 4400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:30.404970 4400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:30.438868 4400 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:43299
I20260809 15:18:30.438946 4534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:43299 every 8 connection(s)
I20260809 15:18:30.440222 4400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:18:30.448629 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 4400
I20260809 15:18:30.448851 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:18:30.449440 15406 external_mini_cluster.cc:1428] Setting key 9fb2bcdc016d3085bc37ec9e4eacf462
Aug 09 15:18:30 dist-test-slave-8w9v krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288710, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.130@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:30.461244 4537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:30.442789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:55851 (local address 127.15.11.190:33175)
0809 15:18:30.443281 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:30.443286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:30.444257 (+ 971us) server_negotiation.cc:407] Connection header received
0809 15:18:30.446063 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:30.446069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:30.446165 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:30.446298 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:30.447916 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:30.449038 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:30.450801 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:30.451353 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:30.451499 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:30.455452 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:30.455457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:30.455460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:30.455492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:30.458087 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:30.458985 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:30.458989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:30.458991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:30.459114 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:30.459619 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:30.459621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:30.459623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:30.459880 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:30.459984 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:30.460837 (+ 853us) server_negotiation.cc:299] Negotiation successful
0809 15:18:30.460991 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":274,"threads_started":1}
I20260809 15:18:30.462623 4538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:30.443310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:33175 (local address 127.15.11.130:55851)
0809 15:18:30.444054 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:30.444092 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:30.445794 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:30.446524 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:30.446533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:30.447080 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:30.447751 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:30.447770 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:30.449217 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:30.449221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:30.450662 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:30.450675 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:30.451581 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:30.451586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:30.451792 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:30.452504 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:18:30.452526 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:30.455235 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:30.458218 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:30.458229 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:30.458244 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:30.458878 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:30.459220 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:30.459228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:30.459235 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:30.459506 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:30.460002 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:30.460013 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:30.460449 (+ 436us) client_negotiation.cc:769] Sending connection context
0809 15:18:30.460733 (+ 284us) client_negotiation.cc:240] Negotiation successful
0809 15:18:30.461259 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":296,"threads_started":1}
I20260809 15:18:30.463999 4535 heartbeater.cc:347] Connected to a master server at 127.15.11.190:33175
I20260809 15:18:30.464329 4535 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:30.465082 4535 heartbeater.cc:511] Master 127.15.11.190:33175 requested a full tablet report, sending...
I20260809 15:18:30.466578 4194 ts_manager.cc:194] Registered new tserver with Master: 6b0c6a9e0d654fc2bb02fdcfa2e01bdb (127.15.11.130:43299)
I20260809 15:18:30.467891 4194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.130@KRBTEST.COM'} at 127.15.11.130:55851
WARNING: no policy specified for kudu/127.15.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
Entry for principal kudu/127.15.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab.
I20260809 15:18:30.517153 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33175
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:40923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:30.856385 4541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:30.856683 4541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:30.856865 4541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:30.863920 4541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:30.864049 4541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:30.864244 4541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:18:30.873623 4541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:40923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.131.keytab
--principal=kudu/127.15.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:33175
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:30.874927 4541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:30.876313 4541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:30.889353 4547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:30.889305 4546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:30.889385 4549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:30.890709 4541 server_base.cc:1034] running on GCE node
I20260809 15:18:30.891273 4541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:30.892200 4541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:30.893422 4541 hybrid_clock.cc:648] HybridClock initialized: now 1786288710893378 us; error 54 us; skew 500 ppm
I20260809 15:18:30.893719 4541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:30 dist-test-slave-8w9v krb5kdc[4147](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288710, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:30.896821 4541 init.cc:377] Logged in from keytab as kudu/127.15.11.131@KRBTEST.COM (short username kudu)
I20260809 15:18:30.898171 4541 webserver.cc:460] Webserver started at http://127.15.11.131:35737/ using document root <none> and password file <none>
I20260809 15:18:30.898820 4541 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:30.898959 4541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:30.899274 4541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:30.902194 4541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "7223239eed6543cba92464f18348946b"
format_stamp: "Formatted at 2026-08-09 15:18:30 on dist-test-slave-8w9v"
server_key: "0d4a44bff15673aadb371a48572e34fd"
server_key_iv: "0a2ee30a7600249b796d7ad3250c8dcd"
server_key_version: "encryptionkey@0"
I20260809 15:18:30.903012 4541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "7223239eed6543cba92464f18348946b"
format_stamp: "Formatted at 2026-08-09 15:18:30 on dist-test-slave-8w9v"
server_key: "0d4a44bff15673aadb371a48572e34fd"
server_key_iv: "0a2ee30a7600249b796d7ad3250c8dcd"
server_key_version: "encryptionkey@0"
I20260809 15:18:30.908053 4541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 15:18:30.911329 4557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:30.912832 4541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 15:18:30.912994 4541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "7223239eed6543cba92464f18348946b"
format_stamp: "Formatted at 2026-08-09 15:18:30 on dist-test-slave-8w9v"
server_key: "0d4a44bff15673aadb371a48572e34fd"
server_key_iv: "0a2ee30a7600249b796d7ad3250c8dcd"
server_key_version: "encryptionkey@0"
I20260809 15:18:30.913213 4541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:30.942341 4541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:30.946748 4541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:30.947113 4541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:30.947137 4394 heartbeater.cc:503] Master 127.15.11.190:33175 was elected leader, sending a full tablet report...
I20260809 15:18:30.948437 4541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:30.950086 4541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:30.950168 4541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:30.950268 4541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:30.950320 4541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:30.986373 4541 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:43425
I20260809 15:18:30.986440 4676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:43425 every 8 connection(s)
I20260809 15:18:30.987918 4541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:18:30.995518 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 4541
I20260809 15:18:30.995730 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:18:30.996243 15406 external_mini_cluster.cc:1428] Setting key 27606e95db7c5980f11d30627d041ed7
Aug 09 15:18:30 dist-test-slave-8w9v krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288710, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.131@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:18:31.008205 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:30.990592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:43767 (local address 127.15.11.190:33175)
0809 15:18:30.991075 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:30.991079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:30.991994 (+ 915us) server_negotiation.cc:407] Connection header received
0809 15:18:30.993873 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:30.993878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:30.993984 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:30.994101 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:30.995735 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:30.996798 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:30.998757 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:30.999280 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:30.999435 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.002772 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:31.002779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:31.002783 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:31.002824 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:31.005220 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.005928 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.005934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.005938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.006064 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.006484 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.006491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.006494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.006765 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:31.006936 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.007571 (+ 635us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.007790 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":239,"threads_started":1}
I20260809 15:18:31.009086 4680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:30.991131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:33175 (local address 127.15.11.131:43767)
0809 15:18:30.991839 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:30.991882 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:30.993620 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:30.994323 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:30.994331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:30.994808 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:30.995520 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:30.995547 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:30.996978 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:30.996985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:30.998619 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:30.998631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:30.999516 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:30.999523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:30.999782 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.000457 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:18:31.000479 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:31.002571 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:31.005366 (+ 2795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:31.005375 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:31.005389 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:31.005794 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:31.006176 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:18:31.006181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:18:31.006184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:18:31.006371 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:18:31.006904 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:31.006912 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:31.007215 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 15:18:31.007472 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 15:18:31.007989 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":273,"threads_started":1}
I20260809 15:18:31.010519 4677 heartbeater.cc:347] Connected to a master server at 127.15.11.190:33175
I20260809 15:18:31.010847 4677 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:31.011677 4677 heartbeater.cc:511] Master 127.15.11.190:33175 requested a full tablet report, sending...
I20260809 15:18:31.013152 4193 ts_manager.cc:194] Registered new tserver with Master: 7223239eed6543cba92464f18348946b (127.15.11.131:43425)
I20260809 15:18:31.014045 4193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.11.131@KRBTEST.COM'} at 127.15.11.131:43767
I20260809 15:18:31.023185 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:18:31.039848 4679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.027256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.15.11.190:33175)
0809 15:18:31.027431 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.027452 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.027858 (+ 406us) server_negotiation.cc:407] Connection header received
0809 15:18:31.028052 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.028057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.028166 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.028267 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:31.029641 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.030355 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.031664 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.032084 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.032253 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.034656 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:31.034666 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:31.034669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:31.034700 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:31.037194 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.037834 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.037841 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.037845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.037971 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.038352 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.038358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.038361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.038693 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:31.038952 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.039293 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.039532 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:18:31.043463 4193 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:18:31.050745 4193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:18:31.099251 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.063341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.15.11.131:43425)
0809 15:18:31.088551 (+ 25210us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.088563 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.090566 (+ 2003us) server_negotiation.cc:407] Connection header received
0809 15:18:31.090673 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.090681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.090947 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.091130 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:31.092110 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.093771 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.096151 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.097942 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.098066 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.098296 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.098671 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.098859 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24976,"thread_start_us":25225,"threads_started":1}
I20260809 15:18:31.099236 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.062617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:43425 (local address 127.0.0.1:45478)
0809 15:18:31.090265 (+ 27648us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.090282 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:31.090412 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:31.091258 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:31.091266 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:31.091303 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:31.091843 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.091857 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.093918 (+ 2061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.093926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.095961 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.095973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.098151 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.098159 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.098409 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.098471 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 15:18:31.098608 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 15:18:31.098800 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27518,"thread_start_us":27349,"threads_started":1}
I20260809 15:18:31.100023 4691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.062918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:43299 (local address 127.0.0.1:39180)
0809 15:18:31.091897 (+ 28979us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.091916 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:31.092103 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:31.092769 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:31.092774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:31.092804 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:31.093178 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.093189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.094831 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.094839 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.097184 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.097199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.099176 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.099183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.099423 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.099468 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 15:18:31.099550 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 15:18:31.099671 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28818,"thread_start_us":26201,"threads_started":1}
I20260809 15:18:31.100433 4693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.062552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.0.0.1:55104)
0809 15:18:31.088470 (+ 25918us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.088536 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:31.088771 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:31.092370 (+ 3599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:31.092374 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:31.092405 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:31.092807 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.092823 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.095196 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.095202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.097442 (+ 2240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.097453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.099492 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.099499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.099679 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.099715 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 15:18:31.099824 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 15:18:31.099960 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25576,"thread_start_us":29050,"threads_started":1}
I20260809 15:18:31.100469 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.063474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.15.11.130:43299)
0809 15:18:31.087538 (+ 24064us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.087551 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.092001 (+ 4450us) server_negotiation.cc:407] Connection header received
0809 15:18:31.092338 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.092346 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.092555 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.092765 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:31.093353 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.094693 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.097333 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.098708 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.098842 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.099018 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.099916 (+ 898us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.100110 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23806,"thread_start_us":24074,"threads_started":1}
I20260809 15:18:31.103389 4609 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2eb5e00502420b81cf26c7a652d24f (DEFAULT_TABLE table=test-table [id=610f36d0950246418680e6da69476d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:31.103466 4465 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2eb5e00502420b81cf26c7a652d24f (DEFAULT_TABLE table=test-table [id=610f36d0950246418680e6da69476d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:31.104514 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.062540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55104 (local address 127.15.11.129:33295)
0809 15:18:31.091671 (+ 29131us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.091680 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.091710 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:18:31.091807 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.091813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.092099 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.092378 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:31.093052 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.095051 (+ 1999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.097585 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.098963 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.099095 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.099322 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.100658 (+ 1336us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.104166 (+ 3508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28913,"thread_start_us":29293,"threads_started":1}
I20260809 15:18:31.104759 4609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2eb5e00502420b81cf26c7a652d24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:31.104775 4465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2eb5e00502420b81cf26c7a652d24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:31.107894 4325 tablet_service.cc:1467] Processing CreateTablet for tablet 4a2eb5e00502420b81cf26c7a652d24f (DEFAULT_TABLE table=test-table [id=610f36d0950246418680e6da69476d78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:31.109261 4325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a2eb5e00502420b81cf26c7a652d24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:31.116021 4696 tablet_bootstrap.cc:492] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Bootstrap starting.
I20260809 15:18:31.119477 4697 tablet_bootstrap.cc:492] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: Bootstrap starting.
I20260809 15:18:31.120281 4696 tablet_bootstrap.cc:654] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:31.121644 4696 log.cc:824] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:31.123126 4697 tablet_bootstrap.cc:654] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:31.124372 4699 tablet_bootstrap.cc:492] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df: Bootstrap starting.
I20260809 15:18:31.124462 4697 log.cc:824] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:31.125072 4696 tablet_bootstrap.cc:492] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: No bootstrap required, opened a new log
I20260809 15:18:31.125415 4696 ts_tablet_manager.cc:1397] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 15:18:31.127490 4699 tablet_bootstrap.cc:654] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:31.127748 4697 tablet_bootstrap.cc:492] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: No bootstrap required, opened a new log
I20260809 15:18:31.127997 4697 ts_tablet_manager.cc:1397] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 15:18:31.128805 4699 log.cc:824] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:31.131155 4699 tablet_bootstrap.cc:492] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df: No bootstrap required, opened a new log
I20260809 15:18:31.131402 4699 ts_tablet_manager.cc:1397] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:18:31.136439 4696 raft_consensus.cc:348] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } }
I20260809 15:18:31.136871 4696 raft_consensus.cc:374] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:31.136974 4696 raft_consensus.cc:729] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b0c6a9e0d654fc2bb02fdcfa2e01bdb, State: Initialized, Role: FOLLOWER
I20260809 15:18:31.137701 4696 consensus_queue.cc:260] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } }
I20260809 15:18:31.138787 4699 raft_consensus.cc:348] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } }
I20260809 15:18:31.138831 4697 raft_consensus.cc:348] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } }
I20260809 15:18:31.139196 4699 raft_consensus.cc:374] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:31.139238 4697 raft_consensus.cc:374] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:31.139302 4699 raft_consensus.cc:729] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a906ccc97fb44e408ad00b4b015ea2df, State: Initialized, Role: FOLLOWER
I20260809 15:18:31.139351 4697 raft_consensus.cc:729] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7223239eed6543cba92464f18348946b, State: Initialized, Role: FOLLOWER
I20260809 15:18:31.139474 4535 heartbeater.cc:503] Master 127.15.11.190:33175 was elected leader, sending a full tablet report...
I20260809 15:18:31.139838 4699 consensus_queue.cc:260] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } }
I20260809 15:18:31.140036 4697 consensus_queue.cc:260] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } }
I20260809 15:18:31.141538 4699 ts_tablet_manager.cc:1428] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260809 15:18:31.142694 4677 heartbeater.cc:503] Master 127.15.11.190:33175 was elected leader, sending a full tablet report...
I20260809 15:18:31.143343 4697 ts_tablet_manager.cc:1428] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260809 15:18:31.144039 4696 ts_tablet_manager.cc:1428] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Time spent starting tablet: real 0.019s user 0.014s sys 0.001s
W20260809 15:18:31.166352 4395 tablet.cc:2369] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:18:31.192390 4536 tablet.cc:2369] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:18:31.239650 4678 tablet.cc:2369] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:31.419574 4702 raft_consensus.cc:482] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:18:31.419879 4702 raft_consensus.cc:504] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } }
I20260809 15:18:31.421751 4702 leader_election.cc:288] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a906ccc97fb44e408ad00b4b015ea2df (127.15.11.129:33295), 7223239eed6543cba92464f18348946b (127.15.11.131:43425)
I20260809 15:18:31.429165 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.422254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:37711 (local address 127.15.11.129:33295)
0809 15:18:31.422508 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.422516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.423122 (+ 606us) server_negotiation.cc:407] Connection header received
0809 15:18:31.423311 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.423317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.423383 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.423481 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:31.424194 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.425203 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.427191 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.428158 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.428309 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.428371 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.428772 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.428911 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 15:18:31.429235 4707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.422219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.15.11.130:37711)
0809 15:18:31.423019 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.423035 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:31.423160 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:31.423508 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:31.423513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:31.423550 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:31.423976 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.423990 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.425328 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.425333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.427022 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.427035 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.428406 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.428413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.428580 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.428621 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 15:18:31.428733 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 15:18:31.428866 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":1,"thread_start_us":476,"threads_started":1}
I20260809 15:18:31.429603 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.422209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:33411 (local address 127.15.11.131:43425)
0809 15:18:31.422492 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.422501 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.423776 (+ 1275us) server_negotiation.cc:407] Connection header received
0809 15:18:31.423941 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.423947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.424027 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.424164 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:31.424715 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.425686 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.427656 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.428674 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.428810 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.428873 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.429288 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.429418 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260809 15:18:31.429783 4708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.422260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:43425 (local address 127.15.11.130:33411)
0809 15:18:31.423673 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.423687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:31.423802 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:31.424186 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:31.424189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:31.424213 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:31.424569 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.424582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.425808 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.425812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.427495 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:31.427508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.428904 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.428910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:31.429099 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.429134 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 15:18:31.429255 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 15:18:31.429402 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"mutex_wait_us":20,"thread_start_us":1066,"threads_started":1}
I20260809 15:18:31.430773 4630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2eb5e00502420b81cf26c7a652d24f" candidate_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7223239eed6543cba92464f18348946b" is_pre_election: true
I20260809 15:18:31.430766 4345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2eb5e00502420b81cf26c7a652d24f" candidate_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a906ccc97fb44e408ad00b4b015ea2df" is_pre_election: true
I20260809 15:18:31.431318 4345 raft_consensus.cc:2454] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0c6a9e0d654fc2bb02fdcfa2e01bdb in term 0.
I20260809 15:18:31.431329 4630 raft_consensus.cc:2454] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b0c6a9e0d654fc2bb02fdcfa2e01bdb in term 0.
I20260809 15:18:31.432009 4417 leader_election.cc:302] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0c6a9e0d654fc2bb02fdcfa2e01bdb, a906ccc97fb44e408ad00b4b015ea2df; no voters:
I20260809 15:18:31.432492 4702 raft_consensus.cc:2781] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:18:31.432655 4702 raft_consensus.cc:482] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:18:31.432804 4702 raft_consensus.cc:3037] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:31.435271 4702 raft_consensus.cc:504] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } }
I20260809 15:18:31.435935 4702 leader_election.cc:288] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [CANDIDATE]: Term 1 election: Requested vote from peers a906ccc97fb44e408ad00b4b015ea2df (127.15.11.129:33295), 7223239eed6543cba92464f18348946b (127.15.11.131:43425)
I20260809 15:18:31.436420 4345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2eb5e00502420b81cf26c7a652d24f" candidate_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a906ccc97fb44e408ad00b4b015ea2df"
I20260809 15:18:31.436477 4630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a2eb5e00502420b81cf26c7a652d24f" candidate_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7223239eed6543cba92464f18348946b"
I20260809 15:18:31.436685 4345 raft_consensus.cc:3037] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:31.436691 4630 raft_consensus.cc:3037] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:31.440081 4630 raft_consensus.cc:2454] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0c6a9e0d654fc2bb02fdcfa2e01bdb in term 1.
I20260809 15:18:31.440081 4345 raft_consensus.cc:2454] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b0c6a9e0d654fc2bb02fdcfa2e01bdb in term 1.
I20260809 15:18:31.440562 4417 leader_election.cc:302] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b0c6a9e0d654fc2bb02fdcfa2e01bdb, a906ccc97fb44e408ad00b4b015ea2df; no voters:
I20260809 15:18:31.440883 4702 raft_consensus.cc:2781] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:31.441390 4702 raft_consensus.cc:686] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 1 LEADER]: Becoming Leader. State: Replica: 6b0c6a9e0d654fc2bb02fdcfa2e01bdb, State: Running, Role: LEADER
I20260809 15:18:31.441911 4702 consensus_queue.cc:237] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } }
I20260809 15:18:31.446285 4192 catalog_manager.cc:5495] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b0c6a9e0d654fc2bb02fdcfa2e01bdb (127.15.11.130). New cstate: current_term: 1 leader_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b0c6a9e0d654fc2bb02fdcfa2e01bdb" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 43299 } health_report { overall_health: HEALTHY } } }
I20260809 15:18:31.465330 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.460164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39192 (local address 127.15.11.130:43299)
0809 15:18:31.460334 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.460339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.460354 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:18:31.460521 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.460524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.460599 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.460710 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:18:31.461050 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.461700 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.463270 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.463637 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.463743 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.464272 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:18:31.464731 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:18:31.464836 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.465061 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.465121 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:18:31.473456 4630 raft_consensus.cc:1261] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [term 1 FOLLOWER]: Refusing update from remote peer 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:31.473466 4345 raft_consensus.cc:1261] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df [term 1 FOLLOWER]: Refusing update from remote peer 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:31.474484 4709 consensus_queue.cc:1035] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7223239eed6543cba92464f18348946b" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 43425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:31.474846 4702 consensus_queue.cc:1035] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a906ccc97fb44e408ad00b4b015ea2df" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 33295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:31.492434 4714 mvcc.cc:205] Tried to move back new op lower bound from 7316638562184867840 to 7316638562070306816. Current Snapshot: MvccSnapshot[applied={T|T < 7316638562184867840 or (T in {7316638562184867840})}]
I20260809 15:18:31.500480 4193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:31.500805 4193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:31.504541 4193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f903ee9c2bf4ea78c933d68d38cb6f9: Sending DeleteTablet for 3 replicas of tablet 4a2eb5e00502420b81cf26c7a652d24f
I20260809 15:18:31.505666 4325 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2eb5e00502420b81cf26c7a652d24f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:31 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:55104
I20260809 15:18:31.505745 4609 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2eb5e00502420b81cf26c7a652d24f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:31 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:45478
I20260809 15:18:31.506435 4465 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a2eb5e00502420b81cf26c7a652d24f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:31 UTC) from {username='kudu', principal='kudu/127.15.11.190@KRBTEST.COM'} at 127.0.0.1:39180
I20260809 15:18:31.507220 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 4258
I20260809 15:18:31.530462 4723 tablet_replica.cc:330] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: stopping tablet replica
I20260809 15:18:31.531124 4723 raft_consensus.cc:2229] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:18:31.531766 4722 tablet_replica.cc:330] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: stopping tablet replica
I20260809 15:18:31.532521 4722 raft_consensus.cc:2229] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:31.533010 4723 raft_consensus.cc:2258] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 15:18:31 dist-test-slave-8w9v krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.129@KRBTEST.COM
I20260809 15:18:31.533879 4722 raft_consensus.cc:2258] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:31.536679 4723 ts_tablet_manager.cc:1905] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:31.536855 4722 ts_tablet_manager.cc:1905] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:31.542963 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.509129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55116 (local address 127.15.11.129:33295)
0809 15:18:31.509376 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.509385 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.519708 (+ 10323us) server_negotiation.cc:407] Connection header received
0809 15:18:31.520251 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.520271 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.520377 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.520477 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:31.522872 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.524558 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.527032 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.527943 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.528061 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.535201 (+ 7140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:31.535217 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:31.535233 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:31.535291 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:31.539101 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.539951 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.539959 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.539963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.540414 (+ 451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.540895 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.540899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.540902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.541431 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:31.541738 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.542353 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.542587 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 15:18:31.544493 4387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:31.544493 4723 ts_tablet_manager.cc:1918] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:31.544714 4722 ts_tablet_manager.cc:1918] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:31.544734 4723 log.cc:1196] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/4a2eb5e00502420b81cf26c7a652d24f
I20260809 15:18:31.544981 4722 log.cc:1196] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/4a2eb5e00502420b81cf26c7a652d24f
I20260809 15:18:31.545477 4723 ts_tablet_manager.cc:1939] T 4a2eb5e00502420b81cf26c7a652d24f P 6b0c6a9e0d654fc2bb02fdcfa2e01bdb: Deleting consensus metadata
I20260809 15:18:31.545668 4722 ts_tablet_manager.cc:1939] T 4a2eb5e00502420b81cf26c7a652d24f P 7223239eed6543cba92464f18348946b: Deleting consensus metadata
I20260809 15:18:31.547735 4180 catalog_manager.cc:4841] TS 7223239eed6543cba92464f18348946b (127.15.11.131:43425): tablet 4a2eb5e00502420b81cf26c7a652d24f (table test-table [id=610f36d0950246418680e6da69476d78]) successfully deleted
I20260809 15:18:31.548236 4178 catalog_manager.cc:4841] TS 6b0c6a9e0d654fc2bb02fdcfa2e01bdb (127.15.11.130:43299): tablet 4a2eb5e00502420b81cf26c7a652d24f (table test-table [id=610f36d0950246418680e6da69476d78]) successfully deleted
I20260809 15:18:31.637596 4721 tablet_replica.cc:330] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df: stopping tablet replica
I20260809 15:18:31.638187 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 4258
I20260809 15:18:31.638228 4721 raft_consensus.cc:2229] T 4a2eb5e00502420b81cf26c7a652d24f P a906ccc97fb44e408ad00b4b015ea2df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:31.660629 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 4400
W20260809 15:18:31.660732 4179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.11.129:33295 (error 108)
I20260809 15:18:31.663305 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.662480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.0.0.1:55130)
0809 15:18:31.662946 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.663091 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:33295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":214,"threads_started":1}
W20260809 15:18:31.663636 4179 catalog_manager.cc:4584] TS a906ccc97fb44e408ad00b4b015ea2df (127.15.11.129:33295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4a2eb5e00502420b81cf26c7a652d24f: Network error: Client connection negotiation failed: client connection to 127.15.11.129:33295: connect: Connection refused (error 111)
Aug 09 15:18:31 dist-test-slave-8w9v krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.130@KRBTEST.COM
I20260809 15:18:31.678645 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.661445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39196 (local address 127.15.11.130:43299)
0809 15:18:31.661640 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.661647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.662014 (+ 367us) server_negotiation.cc:407] Connection header received
0809 15:18:31.662118 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.662123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.662219 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.662338 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:31.664194 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.665326 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.666590 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.667265 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.667420 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.671679 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:31.671698 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:31.671719 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:31.671759 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:31.675709 (+ 3950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.676521 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.676530 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.676534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.676711 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.677067 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.677076 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.677080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.677516 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:31.677872 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.678025 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.678250 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 15:18:31.679370 4529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:31.717258 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.716741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.0.0.1:55144)
0809 15:18:31.716964 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.717091 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:33295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 15:18:31.718385 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.717952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.0.0.1:55146)
0809 15:18:31.718103 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.718228 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:33295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 15:18:31.778913 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.778313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.0.0.1:55160)
0809 15:18:31.778571 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.778691 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:33295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 15:18:31.780071 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.779621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.0.0.1:55168)
0809 15:18:31.779801 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.779892 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:33295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 15:18:31.798122 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 4400
I20260809 15:18:31.821717 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 4541
Aug 09 15:18:31 dist-test-slave-8w9v krb5kdc[4147](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.131@KRBTEST.COM
I20260809 15:18:31.836706 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.822485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45490 (local address 127.15.11.131:43425)
0809 15:18:31.822675 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:31.822679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:31.822697 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:18:31.822862 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:31.822867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:31.822932 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:31.823016 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:31.824458 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.825140 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.826502 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:31.826888 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:31.826977 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:31.831490 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:31.831503 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:31.831518 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:31.831553 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:31.834141 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.834819 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.834827 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.834829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.834957 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:31.835243 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:31.835246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:31.835248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:31.835587 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:31.835812 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:31.835997 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:18:31.836356 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:18:31.837333 4672 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:31.860749 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.860318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.0.0.1:55182)
0809 15:18:31.860508 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.860594 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:33295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 15:18:31.861837 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:31.861465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.0.0.1:55196)
0809 15:18:31.861645 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:31.861708 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:33295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260809 15:18:31.924778 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 4541
I20260809 15:18:31.946470 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 4160
I20260809 15:18:31.947768 4226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:32.059548 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 4160
I20260809 15:18:32.059594 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:32.058373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:33295 (local address 127.0.0.1:55204)
0809 15:18:32.059301 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:32.059415 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.129:33295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
2026-08-09T15:18:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4663 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T15:18:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:32Z Disabled control of system clock
I20260809 15:18:32.150259 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35761
--webserver_interface=127.15.11.190
--webserver_port=0
--builtin_ntp_servers=127.15.11.148:43395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:35761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:18:32.486025 4735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:32.486303 4735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:32.486434 4735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:32.493552 4735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:32.493657 4735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:32.493696 4735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:32.493768 4735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:32.493836 4735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:32.502480 4735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:43395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:35761
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:35761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:32.503671 4735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:32.505441 4735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:32.516952 4740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:32.517010 4741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:32.517046 4743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:32.518126 4735 server_base.cc:1034] running on GCE node
I20260809 15:18:32.518906 4735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:32.520287 4735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:32.521499 4735 hybrid_clock.cc:648] HybridClock initialized: now 1786288712521435 us; error 48 us; skew 500 ppm
I20260809 15:18:32.521821 4735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:32.523945 4735 webserver.cc:460] Webserver started at http://127.15.11.190:39415/ using document root <none> and password file <none>
I20260809 15:18:32.524642 4735 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:32.524751 4735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:32.525067 4735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:32.527830 4735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "6318debac7164943bb9114e768bfa7e5"
format_stamp: "Formatted at 2026-08-09 15:18:32 on dist-test-slave-8w9v"
server_key: "7c4b97fcd24cabbac024ee5deed240f6"
server_key_iv: "dd642ad7ff78eee1fe0410d5406e3197"
server_key_version: "encryptionkey@0"
I20260809 15:18:32.528642 4735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "6318debac7164943bb9114e768bfa7e5"
format_stamp: "Formatted at 2026-08-09 15:18:32 on dist-test-slave-8w9v"
server_key: "7c4b97fcd24cabbac024ee5deed240f6"
server_key_iv: "dd642ad7ff78eee1fe0410d5406e3197"
server_key_version: "encryptionkey@0"
I20260809 15:18:32.533475 4735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:18:32.536593 4749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:32.537858 4735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:18:32.538022 4735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "6318debac7164943bb9114e768bfa7e5"
format_stamp: "Formatted at 2026-08-09 15:18:32 on dist-test-slave-8w9v"
server_key: "7c4b97fcd24cabbac024ee5deed240f6"
server_key_iv: "dd642ad7ff78eee1fe0410d5406e3197"
server_key_version: "encryptionkey@0"
I20260809 15:18:32.538221 4735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:32.568857 4735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:32.569682 4735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:32.569957 4735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:32.588589 4735 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:35761
I20260809 15:18:32.588640 4805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:35761 every 8 connection(s)
I20260809 15:18:32.589920 4735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:32.593773 4806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:32.593755 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 4735
I20260809 15:18:32.593943 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:32.594540 15406 external_mini_cluster.cc:1428] Setting key 5661bdd6f8668190ea0ec477c4f86adc
I20260809 15:18:32.605913 4806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5: Bootstrap starting.
I20260809 15:18:32.608516 4809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:32.596575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.15.11.190:35761)
0809 15:18:32.597581 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:32.597591 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:32.597631 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:18:32.599944 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:32.599966 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:32.600495 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:32.600948 (+ 453us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:32.601475 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:32.602722 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:32.604063 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:32.604750 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:32.604935 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:32.606127 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:32.606135 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:32.606147 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:32.606764 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:32.606839 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:32.606860 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:32.606979 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:32.607201 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 15:18:32.607524 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":361,"threads_started":1}
I20260809 15:18:32.609402 4806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:32.610567 4806 log.cc:824] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:32.613549 4806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5: No bootstrap required, opened a new log
I20260809 15:18:32.620469 4806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6318debac7164943bb9114e768bfa7e5" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35761 } }
I20260809 15:18:32.620857 4806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:32.620965 4806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6318debac7164943bb9114e768bfa7e5, State: Initialized, Role: FOLLOWER
I20260809 15:18:32.621582 4806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6318debac7164943bb9114e768bfa7e5" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35761 } }
I20260809 15:18:32.621829 4806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:32.621978 4806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:32.622146 4806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:32.624271 4806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6318debac7164943bb9114e768bfa7e5" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35761 } }
I20260809 15:18:32.624768 4806 leader_election.cc:302] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6318debac7164943bb9114e768bfa7e5; no voters:
I20260809 15:18:32.625372 4806 leader_election.cc:288] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:32.625537 4811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:32.626511 4811 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [term 1 LEADER]: Becoming Leader. State: Replica: 6318debac7164943bb9114e768bfa7e5, State: Running, Role: LEADER
I20260809 15:18:32.627251 4811 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6318debac7164943bb9114e768bfa7e5" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35761 } }
I20260809 15:18:32.627743 4806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:32.630347 4812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6318debac7164943bb9114e768bfa7e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6318debac7164943bb9114e768bfa7e5" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35761 } } }
I20260809 15:18:32.630826 4812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:32.631425 4813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6318debac7164943bb9114e768bfa7e5. Latest consensus state: current_term: 1 leader_uuid: "6318debac7164943bb9114e768bfa7e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6318debac7164943bb9114e768bfa7e5" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 35761 } } }
I20260809 15:18:32.631801 4813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:32.636464 4820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:32.641623 4820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:32.650142 4820 catalog_manager.cc:1347] Generated new cluster ID: e4b80cfe76db4113b2914820097638a8
I20260809 15:18:32.650261 4820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:32.672997 4820 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:32.674925 4820 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:32.684382 4820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5: Generated new TSK 0
I20260809 15:18:32.685164 4820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:32.702498 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.129:0
--local_ip_for_outbound_sockets=127.15.11.129
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35761
--builtin_ntp_servers=127.15.11.148:43395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:18:33.040962 4831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:33.041278 4831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:33.041429 4831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:33.048833 4831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:33.048954 4831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:33.049122 4831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.129
I20260809 15:18:33.057864 4831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:43395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.11.129
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.129
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:33.059124 4831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:33.060878 4831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:33.073213 4837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:33.073311 4839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:33.073311 4836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:33.074647 4831 server_base.cc:1034] running on GCE node
I20260809 15:18:33.075207 4831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:33.076095 4831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:33.077347 4831 hybrid_clock.cc:648] HybridClock initialized: now 1786288713077292 us; error 51 us; skew 500 ppm
I20260809 15:18:33.077644 4831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:33.079738 4831 webserver.cc:460] Webserver started at http://127.15.11.129:45635/ using document root <none> and password file <none>
I20260809 15:18:33.080469 4831 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:33.080575 4831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:33.080874 4831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:33.083501 4831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/data/instance:
uuid: "af084a00a47e4185b67d4d792381cc90"
format_stamp: "Formatted at 2026-08-09 15:18:33 on dist-test-slave-8w9v"
server_key: "730fb87da01390e304f80ea2d38245bd"
server_key_iv: "3224d78dc66e0c30e703fbfb43451b23"
server_key_version: "encryptionkey@0"
I20260809 15:18:33.084322 4831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance:
uuid: "af084a00a47e4185b67d4d792381cc90"
format_stamp: "Formatted at 2026-08-09 15:18:33 on dist-test-slave-8w9v"
server_key: "730fb87da01390e304f80ea2d38245bd"
server_key_iv: "3224d78dc66e0c30e703fbfb43451b23"
server_key_version: "encryptionkey@0"
I20260809 15:18:33.089102 4831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:18:33.092351 4847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:33.093616 4831 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:18:33.093747 4831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/wal
uuid: "af084a00a47e4185b67d4d792381cc90"
format_stamp: "Formatted at 2026-08-09 15:18:33 on dist-test-slave-8w9v"
server_key: "730fb87da01390e304f80ea2d38245bd"
server_key_iv: "3224d78dc66e0c30e703fbfb43451b23"
server_key_version: "encryptionkey@0"
I20260809 15:18:33.093915 4831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:33.128072 4831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:33.128980 4831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:33.129283 4831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:33.130191 4831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:33.131750 4831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:33.131839 4831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:33.131955 4831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:33.132026 4831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:33.164196 4831 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.129:34295
I20260809 15:18:33.164324 4965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.129:34295 every 8 connection(s)
I20260809 15:18:33.165521 4831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/data/info.pb
I20260809 15:18:33.172149 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 4831
I20260809 15:18:33.172322 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/wal/instance
I20260809 15:18:33.172930 15406 external_mini_cluster.cc:1428] Setting key 592592578a39bac92ed22488f9a86f97
I20260809 15:18:33.175858 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.130:0
--local_ip_for_outbound_sockets=127.15.11.130
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35761
--builtin_ntp_servers=127.15.11.148:43395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:18:33.178660 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:33.167988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.129:47571 (local address 127.15.11.190:35761)
0809 15:18:33.168460 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:33.168466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:33.169602 (+ 1136us) server_negotiation.cc:407] Connection header received
0809 15:18:33.171272 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:33.171276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:33.171339 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:33.171434 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:33.172572 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:33.173613 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:33.175596 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:33.176299 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:33.176428 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:33.177735 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:33.177744 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:33.177751 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:33.177775 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:33.177830 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:33.177850 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:33.177957 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:33.178138 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 15:18:33.178245 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":230,"threads_started":1}
I20260809 15:18:33.179395 4969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:33.168607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35761 (local address 127.15.11.129:47571)
0809 15:18:33.169456 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:33.169491 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:33.171050 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:33.171626 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:33.171634 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:33.171715 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:33.172378 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:33.172417 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:33.173807 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:33.173811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:33.175449 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:33.175462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:33.176523 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:33.176529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:33.176725 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:33.177423 (+ 698us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:18:33.177426 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:33.177464 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:18:33.177542 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:33.177949 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:33.177958 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:33.177964 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:18:33.178098 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 15:18:33.178379 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":270,"threads_started":1}
I20260809 15:18:33.180891 4966 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35761
I20260809 15:18:33.181200 4966 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:33.182094 4966 heartbeater.cc:511] Master 127.15.11.190:35761 requested a full tablet report, sending...
I20260809 15:18:33.184379 4767 ts_manager.cc:194] Registered new tserver with Master: af084a00a47e4185b67d4d792381cc90 (127.15.11.129:34295)
I20260809 15:18:33.185806 4767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.129:47571
W20260809 15:18:33.532907 4970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:33.533198 4970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:33.533345 4970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:33.540459 4970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:33.540580 4970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:33.540778 4970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.130
I20260809 15:18:33.549487 4970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:43395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.11.130
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.130
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:33.550724 4970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:33.552557 4970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:33.565258 4979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:33.565305 4976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:33.565327 4977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:33.566545 4970 server_base.cc:1034] running on GCE node
I20260809 15:18:33.567046 4970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:33.567884 4970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:33.569077 4970 hybrid_clock.cc:648] HybridClock initialized: now 1786288713569020 us; error 45 us; skew 500 ppm
I20260809 15:18:33.569337 4970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:33.571743 4970 webserver.cc:460] Webserver started at http://127.15.11.130:45997/ using document root <none> and password file <none>
I20260809 15:18:33.572434 4970 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:33.572521 4970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:33.572773 4970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:33.575628 4970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/data/instance:
uuid: "81c5774d9bb74ef0803317e077dbea1e"
format_stamp: "Formatted at 2026-08-09 15:18:33 on dist-test-slave-8w9v"
server_key: "7965e13a2386778dc9424b399302e204"
server_key_iv: "77bed161538991e4e4f2f40a0e2a30d6"
server_key_version: "encryptionkey@0"
I20260809 15:18:33.576427 4970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance:
uuid: "81c5774d9bb74ef0803317e077dbea1e"
format_stamp: "Formatted at 2026-08-09 15:18:33 on dist-test-slave-8w9v"
server_key: "7965e13a2386778dc9424b399302e204"
server_key_iv: "77bed161538991e4e4f2f40a0e2a30d6"
server_key_version: "encryptionkey@0"
I20260809 15:18:33.581080 4970 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:18:33.584308 4985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:33.585515 4970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:18:33.585637 4970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/wal
uuid: "81c5774d9bb74ef0803317e077dbea1e"
format_stamp: "Formatted at 2026-08-09 15:18:33 on dist-test-slave-8w9v"
server_key: "7965e13a2386778dc9424b399302e204"
server_key_iv: "77bed161538991e4e4f2f40a0e2a30d6"
server_key_version: "encryptionkey@0"
I20260809 15:18:33.585804 4970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:33.604532 4970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:33.605189 4970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:33.605448 4970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:33.606279 4970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:33.607631 4970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:33.607725 4970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:33.607801 4970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:33.607856 4970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:33.642777 4970 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.130:44571
I20260809 15:18:33.642812 5103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.130:44571 every 8 connection(s)
I20260809 15:18:33.644287 4970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/data/info.pb
I20260809 15:18:33.647419 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 4970
I20260809 15:18:33.647607 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/wal/instance
I20260809 15:18:33.648156 15406 external_mini_cluster.cc:1428] Setting key 534fcb1009ac5da7e3686113b928c82e
I20260809 15:18:33.651424 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.11.131:0
--local_ip_for_outbound_sockets=127.15.11.131
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35761
--builtin_ntp_servers=127.15.11.148:43395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:18:33.656960 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:33.646736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.130:58573 (local address 127.15.11.190:35761)
0809 15:18:33.646913 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:33.646918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:33.648499 (+ 1581us) server_negotiation.cc:407] Connection header received
0809 15:18:33.650480 (+ 1981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:33.650486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:33.650573 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:33.650723 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:33.651779 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:33.652725 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:33.654246 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:33.654661 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:33.654785 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:33.656099 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:33.656103 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:33.656106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:33.656154 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:33.656197 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:33.656210 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:33.656333 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:33.656537 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 15:18:33.656641 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:18:33.657908 5106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:33.647245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35761 (local address 127.15.11.130:58573)
0809 15:18:33.648335 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:33.648378 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:33.650241 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:33.650895 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:33.650903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:33.650982 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:33.651600 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:33.651625 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:33.652877 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:33.652883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:33.654101 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:33.654114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:33.654895 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:33.654905 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:33.655117 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:33.655779 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:18:33.655781 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:33.655826 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:18:33.655899 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:33.656325 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:33.656337 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:33.656347 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 15:18:33.656521 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 15:18:33.656912 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":376,"threads_started":1}
I20260809 15:18:33.659317 5104 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35761
I20260809 15:18:33.659636 5104 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:33.660449 5104 heartbeater.cc:511] Master 127.15.11.190:35761 requested a full tablet report, sending...
I20260809 15:18:33.661876 4767 ts_manager.cc:194] Registered new tserver with Master: 81c5774d9bb74ef0803317e077dbea1e (127.15.11.130:44571)
I20260809 15:18:33.663169 4767 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.130:58573
W20260809 15:18:33.992728 5107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:33.993022 5107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:33.993203 5107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:34.000459 5107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:34.000605 5107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:34.000739 5107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.11.131
I20260809 15:18:34.009919 5107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:43395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.11.131
--webserver_port=0
--tserver_master_addrs=127.15.11.190:35761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.11.131
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:34.011312 5107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:34.013068 5107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:34.026168 5115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:34.026202 5113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:34.026212 5112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:34.026782 5107 server_base.cc:1034] running on GCE node
I20260809 15:18:34.027380 5107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:34.028371 5107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:34.029597 5107 hybrid_clock.cc:648] HybridClock initialized: now 1786288714029551 us; error 56 us; skew 500 ppm
I20260809 15:18:34.029903 5107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:18:34.032531 5107 webserver.cc:460] Webserver started at http://127.15.11.131:39113/ using document root <none> and password file <none>
I20260809 15:18:34.033212 5107 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:34.033336 5107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:34.033649 5107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:34.036532 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/data/instance:
uuid: "b4b3bf5413c947ebb85f88a2c1709209"
format_stamp: "Formatted at 2026-08-09 15:18:34 on dist-test-slave-8w9v"
server_key: "bf4e7b442a8d7d526c17adb46a069fbc"
server_key_iv: "57e76dc4a9057ec4cf87f2f6968271e8"
server_key_version: "encryptionkey@0"
I20260809 15:18:34.037297 5107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance:
uuid: "b4b3bf5413c947ebb85f88a2c1709209"
format_stamp: "Formatted at 2026-08-09 15:18:34 on dist-test-slave-8w9v"
server_key: "bf4e7b442a8d7d526c17adb46a069fbc"
server_key_iv: "57e76dc4a9057ec4cf87f2f6968271e8"
server_key_version: "encryptionkey@0"
I20260809 15:18:34.042115 5107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:18:34.045404 5122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:34.046646 5107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:18:34.046823 5107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/wal
uuid: "b4b3bf5413c947ebb85f88a2c1709209"
format_stamp: "Formatted at 2026-08-09 15:18:34 on dist-test-slave-8w9v"
server_key: "bf4e7b442a8d7d526c17adb46a069fbc"
server_key_iv: "57e76dc4a9057ec4cf87f2f6968271e8"
server_key_version: "encryptionkey@0"
I20260809 15:18:34.047042 5107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:34.059899 5107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:34.060712 5107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:34.061009 5107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:34.061944 5107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:18:34.063297 5107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:18:34.063414 5107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:34.063540 5107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:18:34.063625 5107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:34.097669 5107 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.131:33719
I20260809 15:18:34.097713 5242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.131:33719 every 8 connection(s)
I20260809 15:18:34.099138 5107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/data/info.pb
I20260809 15:18:34.102398 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 5107
I20260809 15:18:34.102600 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/wal/instance
I20260809 15:18:34.103222 15406 external_mini_cluster.cc:1428] Setting key 9564516e00a75778463d879e402cb596
I20260809 15:18:34.112573 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.101836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:54435 (local address 127.15.11.190:35761)
0809 15:18:34.102012 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.102016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.103726 (+ 1710us) server_negotiation.cc:407] Connection header received
0809 15:18:34.106068 (+ 2342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.106073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.106186 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.106328 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:34.107422 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.108300 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.109863 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.110287 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.110418 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.111716 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:34.111721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:34.111724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:34.111741 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:34.111780 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:34.111791 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:34.111885 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.112115 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.112253 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:18:34.113575 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.102349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:35761 (local address 127.15.11.131:54435)
0809 15:18:34.103522 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:34.103571 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:34.105751 (+ 2180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:34.106558 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:34.106566 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:34.106653 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:18:34.107239 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.107255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.108454 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.108458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.109718 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.109729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.110513 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.110520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.110749 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.111383 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:18:34.111385 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:18:34.111424 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:18:34.111509 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:18:34.111886 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:18:34.111900 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:18:34.111910 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 15:18:34.112084 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 15:18:34.112505 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":345,"threads_started":1}
I20260809 15:18:34.114877 5243 heartbeater.cc:347] Connected to a master server at 127.15.11.190:35761
I20260809 15:18:34.115188 5243 heartbeater.cc:464] Registering TS with master...
I20260809 15:18:34.115931 5243 heartbeater.cc:511] Master 127.15.11.190:35761 requested a full tablet report, sending...
I20260809 15:18:34.117584 4765 ts_manager.cc:194] Registered new tserver with Master: b4b3bf5413c947ebb85f88a2c1709209 (127.15.11.131:33719)
I20260809 15:18:34.118517 15406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:18:34.118613 4765 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.11.131:54435
I20260809 15:18:34.128284 4968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.122725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.15.11.190:35761)
0809 15:18:34.122908 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.122912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.123346 (+ 434us) server_negotiation.cc:407] Connection header received
0809 15:18:34.123495 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.123498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.123566 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.123645 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:34.124107 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.124822 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.126126 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.126547 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.126637 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.127612 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:34.127616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:34.127619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:34.127634 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:34.127663 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:34.127674 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:34.127740 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.127864 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.127986 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:18:34.132341 4765 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:18:34.139307 4765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:18:34.165810 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.151763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:34295 (local address 127.0.0.1:49170)
0809 15:18:34.153112 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:34.153162 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:34.153400 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:34.157474 (+ 4074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:34.157482 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:34.157530 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:34.157981 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.157995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.159987 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.159993 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.162552 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.162567 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.164787 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.164797 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.165071 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.165134 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 15:18:34.165259 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 15:18:34.165460 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"mutex_wait_us":1,"thread_start_us":5436,"threads_started":1}
I20260809 15:18:34.167963 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.152967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35882 (local address 127.15.11.131:33719)
0809 15:18:34.157066 (+ 4099us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.157072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.157588 (+ 516us) server_negotiation.cc:407] Connection header received
0809 15:18:34.157947 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.157956 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.158196 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.158375 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:34.159652 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.161243 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.164241 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.166173 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.166336 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.166680 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.167347 (+ 667us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.167543 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3956,"thread_start_us":3781,"threads_started":1}
I20260809 15:18:34.168414 5257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.151812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.15.11.129:34295)
0809 15:18:34.156718 (+ 4906us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.156742 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.156780 (+ 38us) server_negotiation.cc:407] Connection header received
0809 15:18:34.156906 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.156919 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.157230 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.157459 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:34.158196 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.159813 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.162991 (+ 3178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.164565 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.167385 (+ 2820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.167692 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.167757 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.167909 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4684,"thread_start_us":4534,"threads_started":1}
I20260809 15:18:34.168926 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.152242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.131:33719 (local address 127.0.0.1:35882)
0809 15:18:34.157473 (+ 5231us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:34.157491 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:34.157706 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:34.158977 (+ 1271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:34.158982 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:34.159021 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:34.159460 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.159472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.161433 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.161438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.164026 (+ 2588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.164038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.166837 (+ 2799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.166847 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.167109 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.167159 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 15:18:34.168185 (+ 1026us) client_negotiation.cc:240] Negotiation successful
0809 15:18:34.168554 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5085,"mutex_wait_us":498,"thread_start_us":1105,"threads_started":1}
I20260809 15:18:34.172267 4898 tablet_service.cc:1467] Processing CreateTablet for tablet a20538b2e41343e2a758a135d2e461b5 (DEFAULT_TABLE table=test-table [id=9989519300614fb8914add94d7093f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:34.172606 5175 tablet_service.cc:1467] Processing CreateTablet for tablet a20538b2e41343e2a758a135d2e461b5 (DEFAULT_TABLE table=test-table [id=9989519300614fb8914add94d7093f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:34.173736 4898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a20538b2e41343e2a758a135d2e461b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:34.173944 5175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a20538b2e41343e2a758a135d2e461b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:34.175279 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.157942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:44571 (local address 127.0.0.1:35306)
0809 15:18:34.166168 (+ 8226us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:34.166188 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:34.166297 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:34.167134 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:34.167138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:34.167166 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:34.168339 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.168351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.171012 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.171017 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.172761 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.172776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.174538 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.174545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.174762 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.174798 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 15:18:34.174891 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 15:18:34.175024 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8124,"thread_start_us":9341,"threads_started":1}
I20260809 15:18:34.175436 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.158067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35306 (local address 127.15.11.130:44571)
0809 15:18:34.162395 (+ 4328us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.162403 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.166429 (+ 4026us) server_negotiation.cc:407] Connection header received
0809 15:18:34.166530 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.166538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.166755 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.166929 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:34.169294 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.170812 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.172970 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.174369 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.174509 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.174716 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.174965 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.175119 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4161,"thread_start_us":4048,"threads_started":1}
I20260809 15:18:34.179093 5036 tablet_service.cc:1467] Processing CreateTablet for tablet a20538b2e41343e2a758a135d2e461b5 (DEFAULT_TABLE table=test-table [id=9989519300614fb8914add94d7093f98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:18:34.180522 5036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a20538b2e41343e2a758a135d2e461b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:34.185729 5262 tablet_bootstrap.cc:492] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: Bootstrap starting.
I20260809 15:18:34.189109 4966 heartbeater.cc:503] Master 127.15.11.190:35761 was elected leader, sending a full tablet report...
I20260809 15:18:34.189109 5263 tablet_bootstrap.cc:492] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: Bootstrap starting.
I20260809 15:18:34.190441 5262 tablet_bootstrap.cc:654] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:34.191828 5262 log.cc:824] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:34.192924 5263 tablet_bootstrap.cc:654] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:34.194300 5263 log.cc:824] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:34.196408 5266 tablet_bootstrap.cc:492] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: Bootstrap starting.
I20260809 15:18:34.198028 5263 tablet_bootstrap.cc:492] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: No bootstrap required, opened a new log
I20260809 15:18:34.198066 5262 tablet_bootstrap.cc:492] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: No bootstrap required, opened a new log
I20260809 15:18:34.198289 5263 ts_tablet_manager.cc:1397] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 15:18:34.198329 5262 ts_tablet_manager.cc:1397] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260809 15:18:34.200064 5266 tablet_bootstrap.cc:654] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:34.201341 5266 log.cc:824] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:34.204569 5266 tablet_bootstrap.cc:492] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: No bootstrap required, opened a new log
I20260809 15:18:34.204851 5266 ts_tablet_manager.cc:1397] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 15:18:34.206189 5263 raft_consensus.cc:348] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } }
I20260809 15:18:34.206565 5263 raft_consensus.cc:374] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:34.206689 5263 raft_consensus.cc:729] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af084a00a47e4185b67d4d792381cc90, State: Initialized, Role: FOLLOWER
I20260809 15:18:34.207311 5263 consensus_queue.cc:260] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } }
I20260809 15:18:34.208964 5263 ts_tablet_manager.cc:1428] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260809 15:18:34.210542 5262 raft_consensus.cc:348] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } }
I20260809 15:18:34.210991 5262 raft_consensus.cc:374] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:34.211092 5262 raft_consensus.cc:729] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4b3bf5413c947ebb85f88a2c1709209, State: Initialized, Role: FOLLOWER
I20260809 15:18:34.211710 5262 consensus_queue.cc:260] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } }
I20260809 15:18:34.212731 5243 heartbeater.cc:503] Master 127.15.11.190:35761 was elected leader, sending a full tablet report...
I20260809 15:18:34.213351 5262 ts_tablet_manager.cc:1428] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: Time spent starting tablet: real 0.015s user 0.006s sys 0.007s
I20260809 15:18:34.215001 5266 raft_consensus.cc:348] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } }
I20260809 15:18:34.215461 5266 raft_consensus.cc:374] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:34.215556 5266 raft_consensus.cc:729] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81c5774d9bb74ef0803317e077dbea1e, State: Initialized, Role: FOLLOWER
I20260809 15:18:34.216215 5266 consensus_queue.cc:260] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } }
I20260809 15:18:34.217346 5104 heartbeater.cc:503] Master 127.15.11.190:35761 was elected leader, sending a full tablet report...
I20260809 15:18:34.217797 5266 ts_tablet_manager.cc:1428] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260809 15:18:34.351046 5244 tablet.cc:2369] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:18:34.396382 5105 tablet.cc:2369] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:18:34.418228 4967 tablet.cc:2369] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:18:34.442718 5269 raft_consensus.cc:482] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:18:34.443079 5269 raft_consensus.cc:504] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } }
I20260809 15:18:34.444980 5269 leader_election.cc:288] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af084a00a47e4185b67d4d792381cc90 (127.15.11.129:34295), 81c5774d9bb74ef0803317e077dbea1e (127.15.11.130:44571)
I20260809 15:18:34.452201 5257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.445347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:33151 (local address 127.15.11.129:34295)
0809 15:18:34.445557 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.445566 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.445738 (+ 172us) server_negotiation.cc:407] Connection header received
0809 15:18:34.446009 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.446017 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.446127 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.446233 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:34.446888 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.447938 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.449952 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.451062 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.451238 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.451310 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.451700 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.451901 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:18:34.452456 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.446001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.11.131:56243 (local address 127.15.11.130:44571)
0809 15:18:34.446201 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.446209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.446696 (+ 487us) server_negotiation.cc:407] Connection header received
0809 15:18:34.446874 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.446878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.446953 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.447018 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:18:34.447652 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.448559 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.450350 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.451517 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.451692 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.451757 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.452030 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.452192 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:18:34.452206 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.445404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.129:34295 (local address 127.15.11.131:33151)
0809 15:18:34.445636 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:34.445656 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:34.445788 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:34.446402 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:34.446407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:34.446440 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:34.446689 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.446701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.448074 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.448080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.449743 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.449758 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.451281 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.451286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.451415 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.451462 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 15:18:34.451657 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 15:18:34.451818 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 15:18:34.452551 5271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.445808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.130:44571 (local address 127.15.11.131:56243)
0809 15:18:34.446557 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0809 15:18:34.446590 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 15:18:34.446723 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:18:34.447079 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:18:34.447083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:18:34.447108 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:18:34.447480 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.447493 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.448709 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.448713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.450171 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:18:34.450183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.451712 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.451718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:18:34.451876 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.451898 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:18:34.452002 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:18:34.452117 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":409,"threads_started":1}
I20260809 15:18:34.453737 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a20538b2e41343e2a758a135d2e461b5" candidate_uuid: "b4b3bf5413c947ebb85f88a2c1709209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af084a00a47e4185b67d4d792381cc90" is_pre_election: true
I20260809 15:18:34.453737 5056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a20538b2e41343e2a758a135d2e461b5" candidate_uuid: "b4b3bf5413c947ebb85f88a2c1709209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c5774d9bb74ef0803317e077dbea1e" is_pre_election: true
I20260809 15:18:34.454344 5056 raft_consensus.cc:2454] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4b3bf5413c947ebb85f88a2c1709209 in term 0.
I20260809 15:18:34.454344 4918 raft_consensus.cc:2454] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4b3bf5413c947ebb85f88a2c1709209 in term 0.
I20260809 15:18:34.455034 5124 leader_election.cc:302] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81c5774d9bb74ef0803317e077dbea1e, b4b3bf5413c947ebb85f88a2c1709209; no voters:
I20260809 15:18:34.455363 5269 raft_consensus.cc:2781] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:18:34.455542 5269 raft_consensus.cc:482] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:18:34.455657 5269 raft_consensus.cc:3037] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:34.458134 5269 raft_consensus.cc:504] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } }
I20260809 15:18:34.458861 5269 leader_election.cc:288] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [CANDIDATE]: Term 1 election: Requested vote from peers af084a00a47e4185b67d4d792381cc90 (127.15.11.129:34295), 81c5774d9bb74ef0803317e077dbea1e (127.15.11.130:44571)
I20260809 15:18:34.459304 4918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a20538b2e41343e2a758a135d2e461b5" candidate_uuid: "b4b3bf5413c947ebb85f88a2c1709209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af084a00a47e4185b67d4d792381cc90"
I20260809 15:18:34.459383 5056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a20538b2e41343e2a758a135d2e461b5" candidate_uuid: "b4b3bf5413c947ebb85f88a2c1709209" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81c5774d9bb74ef0803317e077dbea1e"
I20260809 15:18:34.459499 4918 raft_consensus.cc:3037] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:34.459626 5056 raft_consensus.cc:3037] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:34.462101 4918 raft_consensus.cc:2454] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4b3bf5413c947ebb85f88a2c1709209 in term 1.
I20260809 15:18:34.462163 5056 raft_consensus.cc:2454] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4b3bf5413c947ebb85f88a2c1709209 in term 1.
I20260809 15:18:34.462503 5124 leader_election.cc:302] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af084a00a47e4185b67d4d792381cc90, b4b3bf5413c947ebb85f88a2c1709209; no voters:
I20260809 15:18:34.462777 5269 raft_consensus.cc:2781] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:34.463227 5269 raft_consensus.cc:686] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 1 LEADER]: Becoming Leader. State: Replica: b4b3bf5413c947ebb85f88a2c1709209, State: Running, Role: LEADER
I20260809 15:18:34.463697 5269 consensus_queue.cc:237] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } }
I20260809 15:18:34.468534 4765 catalog_manager.cc:5495] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4b3bf5413c947ebb85f88a2c1709209 (127.15.11.131). New cstate: current_term: 1 leader_uuid: "b4b3bf5413c947ebb85f88a2c1709209" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4b3bf5413c947ebb85f88a2c1709209" member_type: VOTER last_known_addr { host: "127.15.11.131" port: 33719 } health_report { overall_health: HEALTHY } } }
I20260809 15:18:34.528975 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.523598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.15.11.131:33719)
0809 15:18:34.523811 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.523817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.523837 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:18:34.523967 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.523971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.524059 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.524235 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:18:34.524641 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.525371 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.526758 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.527181 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.527313 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.527834 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:18:34.528409 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:18:34.528578 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.528640 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.528719 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:18:34.537137 5056 raft_consensus.cc:1261] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [term 1 FOLLOWER]: Refusing update from remote peer b4b3bf5413c947ebb85f88a2c1709209: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:34.537137 4918 raft_consensus.cc:1261] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [term 1 FOLLOWER]: Refusing update from remote peer b4b3bf5413c947ebb85f88a2c1709209: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:18:34.538084 5272 consensus_queue.cc:1035] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81c5774d9bb74ef0803317e077dbea1e" member_type: VOTER last_known_addr { host: "127.15.11.130" port: 44571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:34.538463 5269 consensus_queue.cc:1035] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af084a00a47e4185b67d4d792381cc90" member_type: VOTER last_known_addr { host: "127.15.11.129" port: 34295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:18:34.550779 5277 mvcc.cc:205] Tried to move back new op lower bound from 7316638574733910016 to 7316638574446977024. Current Snapshot: MvccSnapshot[applied={T|T < 7316638574733910016}]
I20260809 15:18:34.562239 4765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:34.562482 4765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:18:34.565795 4765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6318debac7164943bb9114e768bfa7e5: Sending DeleteTablet for 3 replicas of tablet a20538b2e41343e2a758a135d2e461b5
I20260809 15:18:34.566965 5036 tablet_service.cc:1514] Processing DeleteTablet for tablet a20538b2e41343e2a758a135d2e461b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:34 UTC) from {username='slave'} at 127.0.0.1:35306
I20260809 15:18:34.566938 5175 tablet_service.cc:1514] Processing DeleteTablet for tablet a20538b2e41343e2a758a135d2e461b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:34 UTC) from {username='slave'} at 127.0.0.1:35882
I20260809 15:18:34.567008 4898 tablet_service.cc:1514] Processing DeleteTablet for tablet a20538b2e41343e2a758a135d2e461b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:18:34 UTC) from {username='slave'} at 127.0.0.1:49170
I20260809 15:18:34.568902 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 4831
I20260809 15:18:34.578408 5287 tablet_replica.cc:330] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: stopping tablet replica
I20260809 15:18:34.578938 5287 raft_consensus.cc:2229] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:34.579288 5288 tablet_replica.cc:330] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: stopping tablet replica
I20260809 15:18:34.579461 5287 raft_consensus.cc:2258] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:34.579701 5288 raft_consensus.cc:2229] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:18:34.580296 5288 raft_consensus.cc:2258] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:34.580410 5286 tablet_replica.cc:330] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: stopping tablet replica
I20260809 15:18:34.580792 5286 raft_consensus.cc:2229] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:18:34.581453 5286 raft_consensus.cc:2258] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:18:34.582059 5287 ts_tablet_manager.cc:1905] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:34.582677 5288 ts_tablet_manager.cc:1905] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:34.583338 5286 ts_tablet_manager.cc:1905] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:18:34.589118 5287 ts_tablet_manager.cc:1918] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:34.589275 5287 log.cc:1196] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-1/wal/wals/a20538b2e41343e2a758a135d2e461b5
I20260809 15:18:34.589818 5287 ts_tablet_manager.cc:1939] T a20538b2e41343e2a758a135d2e461b5 P 81c5774d9bb74ef0803317e077dbea1e: Deleting consensus metadata
I20260809 15:18:34.590629 5286 ts_tablet_manager.cc:1918] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:34.590776 5286 log.cc:1196] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-2/wal/wals/a20538b2e41343e2a758a135d2e461b5
I20260809 15:18:34.591274 5286 ts_tablet_manager.cc:1939] T a20538b2e41343e2a758a135d2e461b5 P b4b3bf5413c947ebb85f88a2c1709209: Deleting consensus metadata
I20260809 15:18:34.591708 4751 catalog_manager.cc:4841] TS 81c5774d9bb74ef0803317e077dbea1e (127.15.11.130:44571): tablet a20538b2e41343e2a758a135d2e461b5 (table test-table [id=9989519300614fb8914add94d7093f98]) successfully deleted
I20260809 15:18:34.592984 4752 catalog_manager.cc:4841] TS b4b3bf5413c947ebb85f88a2c1709209 (127.15.11.131:33719): tablet a20538b2e41343e2a758a135d2e461b5 (table test-table [id=9989519300614fb8914add94d7093f98]) successfully deleted
I20260809 15:18:34.594686 5288 ts_tablet_manager.cc:1918] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:18:34.594851 5288 log.cc:1196] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: Deleting WAL directory at /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786288392170773-15406-0/minicluster-data/ts-0/wal/wals/a20538b2e41343e2a758a135d2e461b5
I20260809 15:18:34.595381 5288 ts_tablet_manager.cc:1939] T a20538b2e41343e2a758a135d2e461b5 P af084a00a47e4185b67d4d792381cc90: Deleting consensus metadata
I20260809 15:18:34.596873 4751 catalog_manager.cc:4841] TS af084a00a47e4185b67d4d792381cc90 (127.15.11.129:34295): tablet a20538b2e41343e2a758a135d2e461b5 (table test-table [id=9989519300614fb8914add94d7093f98]) successfully deleted
I20260809 15:18:34.600364 5257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.570816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49186 (local address 127.15.11.129:34295)
0809 15:18:34.571058 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.571068 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.588991 (+ 17923us) server_negotiation.cc:407] Connection header received
0809 15:18:34.589542 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.589548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.589637 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.589701 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:34.590359 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.591456 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.593496 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.594154 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.594270 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.599024 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:34.599032 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:34.599046 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:34.599580 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:34.599625 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:34.599644 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:34.599762 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.599897 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.600030 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 15:18:34.601105 4961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:34.689327 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 4831
I20260809 15:18:34.717820 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 4970
I20260809 15:18:34.725020 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.718617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35318 (local address 127.15.11.130:44571)
0809 15:18:34.718799 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.718805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.718881 (+ 76us) server_negotiation.cc:407] Connection header received
0809 15:18:34.719106 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.719116 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.719215 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.719321 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:34.719745 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.720562 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.721822 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.722243 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.722365 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.723475 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:34.723483 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:34.723498 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:34.724100 (+ 602us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:34.724311 (+ 211us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:34.724341 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:34.724466 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.724584 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.724702 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:18:34.725633 5098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:34.814414 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 4970
I20260809 15:18:34.841921 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5107
I20260809 15:18:34.849432 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:34.842694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35906 (local address 127.15.11.131:33719)
0809 15:18:34.842858 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:34.842865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:34.842967 (+ 102us) server_negotiation.cc:407] Connection header received
0809 15:18:34.843133 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:34.843139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:34.843244 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:34.843354 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:34.843703 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.844902 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.846066 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:34.846723 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:34.846867 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:34.847838 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:34.847851 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:34.847869 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:34.848658 (+ 789us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:34.848725 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:34.848751 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:34.848872 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:34.848958 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 15:18:34.849057 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:18:34.850165 5238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:34.967307 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 5107
I20260809 15:18:34.991596 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 4735
I20260809 15:18:34.992682 4800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:35.099900 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 4735
2026-08-09T15:18:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3040 ms)
[----------] 6 tests from EncryptionPolicyTest (22545 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:35 dist-test-slave-8w9v krb5kdc[5298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:35 dist-test-slave-8w9v krb5kdc[5298](info): set up 2 sockets
Aug 09 15:18:35 dist-test-slave-8w9v krb5kdc[5298](info): commencing operation
krb5kdc: starting...
W20260809 15:18:36.241592 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.006s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:36 dist-test-slave-8w9v krb5kdc[5298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:36.517406 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:36.851615 5312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:36.851864 5312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:36.851950 5312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:36.858770 5312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:36.858870 5312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:36.858922 5312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:36.858984 5312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:36.859027 5312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:36.868285 5312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35121
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:36.869401 5312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:36.870617 5312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:36.882120 5317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:36.882164 5318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:36.882366 5320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:36.883217 5312 server_base.cc:1034] running on GCE node
I20260809 15:18:36.883942 5312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:36.885375 5312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:36.886615 5312 hybrid_clock.cc:648] HybridClock initialized: now 1786288716886572 us; error 59 us; skew 500 ppm
I20260809 15:18:36.886953 5312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:36 dist-test-slave-8w9v krb5kdc[5298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288716, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:36.890043 5312 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:36.891408 5312 webserver.cc:460] Webserver started at http://127.0.0.1:43157/ using document root <none> and password file <none>
I20260809 15:18:36.892053 5312 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:36.892213 5312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:36.892508 5312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:36.895328 5312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "25e196af40414f2484b02d9dea36c47d"
format_stamp: "Formatted at 2026-08-09 15:18:36 on dist-test-slave-8w9v"
server_key: "ac76575bfdd2b9188b3c139d60528b62"
server_key_iv: "81125ad3d9a62cd7c392b70d57e40f17"
server_key_version: "encryptionkey@0"
I20260809 15:18:36.896168 5312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "25e196af40414f2484b02d9dea36c47d"
format_stamp: "Formatted at 2026-08-09 15:18:36 on dist-test-slave-8w9v"
server_key: "ac76575bfdd2b9188b3c139d60528b62"
server_key_iv: "81125ad3d9a62cd7c392b70d57e40f17"
server_key_version: "encryptionkey@0"
I20260809 15:18:36.900975 5312 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:18:36.904321 5327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:36.905519 5312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:18:36.905686 5312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "25e196af40414f2484b02d9dea36c47d"
format_stamp: "Formatted at 2026-08-09 15:18:36 on dist-test-slave-8w9v"
server_key: "ac76575bfdd2b9188b3c139d60528b62"
server_key_iv: "81125ad3d9a62cd7c392b70d57e40f17"
server_key_version: "encryptionkey@0"
I20260809 15:18:36.905872 5312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:36.925175 5312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:36.929092 5312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:36.929411 5312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:36.948609 5312 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35121
I20260809 15:18:36.948649 5382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35121 every 8 connection(s)
I20260809 15:18:36.949944 5312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:36.953693 5383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:36.954550 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 5312
I20260809 15:18:36.954752 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:36.955394 15406 external_mini_cluster.cc:1428] Setting key 865c7d71d7f89332a11639b74a78a148
I20260809 15:18:36.967655 5383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d: Bootstrap starting.
Aug 09 15:18:36 dist-test-slave-8w9v krb5kdc[5298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:18:36.971487 5383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:36.972589 5383 log.cc:824] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:36.975358 5383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d: No bootstrap required, opened a new log
I20260809 15:18:36.979861 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:36.957588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59105 (local address 127.0.0.1:35121)
0809 15:18:36.958537 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:36.958553 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:36.958597 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:18:36.960384 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:36.960408 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:36.960816 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:36.961155 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:36.962880 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:36.964375 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:36.966044 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:36.966838 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:36.966971 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:36.972037 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:36.972045 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:36.972059 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:36.972100 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:36.975466 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:36.976328 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:36.976339 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:36.976345 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:36.976526 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:36.977049 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:36.977053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:36.977055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:36.977621 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:36.977993 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:36.978549 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 15:18:36.978853 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":710,"thread_start_us":463,"threads_started":1}
I20260809 15:18:36.983498 5383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e196af40414f2484b02d9dea36c47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35121 } }
I20260809 15:18:36.983955 5383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:36.984071 5383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e196af40414f2484b02d9dea36c47d, State: Initialized, Role: FOLLOWER
I20260809 15:18:36.984257 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:36.984680 5383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e196af40414f2484b02d9dea36c47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35121 } }
I20260809 15:18:36.984912 5383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:36.985041 5383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:36.985204 5383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:36.987322 5383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e196af40414f2484b02d9dea36c47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35121 } }
I20260809 15:18:36.987793 5383 leader_election.cc:302] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25e196af40414f2484b02d9dea36c47d; no voters:
I20260809 15:18:36.988478 5383 leader_election.cc:288] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:36.988583 5392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:36.990691 5392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [term 1 LEADER]: Becoming Leader. State: Replica: 25e196af40414f2484b02d9dea36c47d, State: Running, Role: LEADER
I20260809 15:18:36.991003 5383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:36.991324 5392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e196af40414f2484b02d9dea36c47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35121 } }
I20260809 15:18:36.995255 5393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25e196af40414f2484b02d9dea36c47d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e196af40414f2484b02d9dea36c47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35121 } } }
I20260809 15:18:36.995340 5394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25e196af40414f2484b02d9dea36c47d. Latest consensus state: current_term: 1 leader_uuid: "25e196af40414f2484b02d9dea36c47d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25e196af40414f2484b02d9dea36c47d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35121 } } }
I20260809 15:18:36.995764 5393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:36.995770 5394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:37.009550 5401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:37.015342 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:36.989973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60429 (local address 127.0.0.1:35121)
0809 15:18:36.990155 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:36.990160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:36.990182 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:18:36.990645 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:36.990653 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:36.990757 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:36.990842 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:36.992756 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:36.994030 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:36.997659 (+ 3629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:36.998700 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:36.998823 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:37.003472 (+ 4649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:37.003482 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:37.003489 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:37.003546 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:37.009611 (+ 6065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:37.010998 (+ 1387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:37.011006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:37.011012 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:37.011247 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:37.012037 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:37.012045 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:37.012051 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:37.012687 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:37.013051 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:37.014521 (+ 1470us) server_negotiation.cc:299] Negotiation successful
0809 15:18:37.014766 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:18:37.015748 5401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:37.025703 5401 catalog_manager.cc:1347] Generated new cluster ID: 615cc03bec1b4c33b36a4af68dae0093
I20260809 15:18:37.025832 5401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:37.070868 5401 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:37.072270 5401 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:37.079207 5401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25e196af40414f2484b02d9dea36c47d: Generated new TSK 0
I20260809 15:18:37.080307 5401 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:37.094663 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5312
I20260809 15:18:37.096026 5378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:37.190155 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 5312
W20260809 15:18:37.212299 5296 connection.cc:535] client connection to 127.0.0.1:35121 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35121): Connection reset by peer (error 104)
2026-08-09T15:18:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2083 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:37 dist-test-slave-8w9v krb5kdc[5415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:37 dist-test-slave-8w9v krb5kdc[5415](info): set up 2 sockets
Aug 09 15:18:37 dist-test-slave-8w9v krb5kdc[5415](info): commencing operation
krb5kdc: starting...
W20260809 15:18:38.348325 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:38 dist-test-slave-8w9v krb5kdc[5415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:38.633646 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43943
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:38.972573 5428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:38.972846 5428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:38.972935 5428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:38.979513 5428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:38.979609 5428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:38.979655 5428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:38.979707 5428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:38.979740 5428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:38.989003 5428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43943
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:38.990178 5428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:38.991393 5428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:39.003111 5435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:39.003214 5434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:39.003189 5437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:39.004249 5428 server_base.cc:1034] running on GCE node
I20260809 15:18:39.004973 5428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:39.006349 5428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:39.007591 5428 hybrid_clock.cc:648] HybridClock initialized: now 1786288719007545 us; error 48 us; skew 500 ppm
I20260809 15:18:39.007938 5428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:39 dist-test-slave-8w9v krb5kdc[5415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288719, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:39.011000 5428 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:39.012514 5428 webserver.cc:460] Webserver started at http://127.0.0.1:39811/ using document root <none> and password file <none>
I20260809 15:18:39.013162 5428 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:39.013306 5428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:39.013612 5428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:39.016521 5428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "930c94e04c754affa7980214329f4acd"
format_stamp: "Formatted at 2026-08-09 15:18:39 on dist-test-slave-8w9v"
server_key: "a2bc35c23cafba25f98ca9a0b34407e6"
server_key_iv: "3639c11175cdbeea8773a45176ebf352"
server_key_version: "encryptionkey@0"
I20260809 15:18:39.017300 5428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "930c94e04c754affa7980214329f4acd"
format_stamp: "Formatted at 2026-08-09 15:18:39 on dist-test-slave-8w9v"
server_key: "a2bc35c23cafba25f98ca9a0b34407e6"
server_key_iv: "3639c11175cdbeea8773a45176ebf352"
server_key_version: "encryptionkey@0"
I20260809 15:18:39.022351 5428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260809 15:18:39.025802 5446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:39.027015 5428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:18:39.027236 5428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "930c94e04c754affa7980214329f4acd"
format_stamp: "Formatted at 2026-08-09 15:18:39 on dist-test-slave-8w9v"
server_key: "a2bc35c23cafba25f98ca9a0b34407e6"
server_key_iv: "3639c11175cdbeea8773a45176ebf352"
server_key_version: "encryptionkey@0"
I20260809 15:18:39.027439 5428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:39.037851 5428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:39.041599 5428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:39.041880 5428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:39.060302 5428 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43943
I20260809 15:18:39.060289 5500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43943 every 8 connection(s)
I20260809 15:18:39.061679 5428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:39.063237 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 5428
I20260809 15:18:39.063454 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:39.064028 15406 external_mini_cluster.cc:1428] Setting key 88961fe81685900fd3a6838a996e2dcc
I20260809 15:18:39.066181 5501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:39.077589 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:39.065953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57707 (local address 127.0.0.1:43943)
0809 15:18:39.066783 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:39.066794 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:39.066833 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:18:39.068544 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:39.068565 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:39.068921 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:39.069226 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:39.069837 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:39.071282 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:39.073207 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:39.073851 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:39.073987 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:39.075269 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:39.075277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:39.075289 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:39.075854 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:39.075928 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:39.075953 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:39.076104 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:39.076475 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 15:18:39.076668 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":428,"threads_started":1}
I20260809 15:18:39.079131 5501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd: Bootstrap starting.
I20260809 15:18:39.081530 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:39.082479 5501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:39.083438 5501 log.cc:824] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:39.086474 5501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd: No bootstrap required, opened a new log
I20260809 15:18:39.092018 5504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:39.085766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39961 (local address 127.0.0.1:43943)
0809 15:18:39.085907 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:39.085911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:39.086363 (+ 452us) server_negotiation.cc:407] Connection header received
0809 15:18:39.086576 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:39.086584 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:39.086655 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:39.086752 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:39.087278 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:39.088101 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:39.089477 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:39.089946 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:39.090077 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:39.091248 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:39.091255 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:39.091258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:39.091292 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:39.091331 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:39.091345 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:39.091487 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:39.091675 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 15:18:39.091738 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:18:39.094370 5501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "930c94e04c754affa7980214329f4acd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43943 } }
I20260809 15:18:39.094797 5501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:39.094908 5501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 930c94e04c754affa7980214329f4acd, State: Initialized, Role: FOLLOWER
I20260809 15:18:39.095474 5501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "930c94e04c754affa7980214329f4acd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43943 } }
I20260809 15:18:39.095757 5501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:39.095933 5501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:39.096167 5501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:39.098485 5501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "930c94e04c754affa7980214329f4acd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43943 } }
I20260809 15:18:39.098997 5501 leader_election.cc:302] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 930c94e04c754affa7980214329f4acd; no voters:
I20260809 15:18:39.099692 5501 leader_election.cc:288] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:39.099797 5513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:39.100587 5513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [term 1 LEADER]: Becoming Leader. State: Replica: 930c94e04c754affa7980214329f4acd, State: Running, Role: LEADER
I20260809 15:18:39.101115 5513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "930c94e04c754affa7980214329f4acd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43943 } }
I20260809 15:18:39.101796 5501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:39.103900 5515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "930c94e04c754affa7980214329f4acd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "930c94e04c754affa7980214329f4acd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43943 } } }
I20260809 15:18:39.103924 5514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 930c94e04c754affa7980214329f4acd. Latest consensus state: current_term: 1 leader_uuid: "930c94e04c754affa7980214329f4acd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "930c94e04c754affa7980214329f4acd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43943 } } }
I20260809 15:18:39.104353 5515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:39.104353 5514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:39.106725 5518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:39.111874 5518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:39.120535 5518 catalog_manager.cc:1347] Generated new cluster ID: 09e15efd1b644669a79d125cb467efa2
I20260809 15:18:39.120643 5518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:39.138155 5518 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:39.139897 5518 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:39.149091 5518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 930c94e04c754affa7980214329f4acd: Generated new TSK 0
I20260809 15:18:39.150018 5518 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:39.155876 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5428
W20260809 15:18:39.157447 5495 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57707
W20260809 15:18:39.158175 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5428: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:18:39.158372 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 5428
W20260809 15:18:39.179657 5413 connection.cc:535] client connection to 127.0.0.1:43943 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43943): Connection reset by peer (error 104)
2026-08-09T15:18:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1967 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:39 dist-test-slave-8w9v krb5kdc[5535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:39 dist-test-slave-8w9v krb5kdc[5535](info): set up 2 sockets
Aug 09 15:18:39 dist-test-slave-8w9v krb5kdc[5535](info): commencing operation
krb5kdc: starting...
W20260809 15:18:40.300273 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.007s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:40 dist-test-slave-8w9v krb5kdc[5535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:40.585569 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45119
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:40.924043 5549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:40.924396 5549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:40.924489 5549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:40.931685 5549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:40.931784 5549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:40.931833 5549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:40.931893 5549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:40.931941 5549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:40.941963 5549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45119
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:40.943140 5549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:40.944514 5549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:40.956573 5554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:40.956596 5557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:40.956620 5555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:40.957435 5549 server_base.cc:1034] running on GCE node
I20260809 15:18:40.958174 5549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:40.959555 5549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:40.960796 5549 hybrid_clock.cc:648] HybridClock initialized: now 1786288720960737 us; error 50 us; skew 500 ppm
I20260809 15:18:40.961162 5549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:40 dist-test-slave-8w9v krb5kdc[5535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288720, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:40.970608 5549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:40.972862 5549 webserver.cc:460] Webserver started at http://127.0.0.1:38393/ using document root <none> and password file <none>
I20260809 15:18:40.973541 5549 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:40.973683 5549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:40.974000 5549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:40.977065 5549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "1ff3981347d741bfb39b9b89fc9b1769"
format_stamp: "Formatted at 2026-08-09 15:18:40 on dist-test-slave-8w9v"
server_key: "87021120296bdeb4d088d0557cbcbe45"
server_key_iv: "67ed553ed1d923b41cb2c2736d5bfef2"
server_key_version: "encryptionkey@0"
I20260809 15:18:40.977833 5549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "1ff3981347d741bfb39b9b89fc9b1769"
format_stamp: "Formatted at 2026-08-09 15:18:40 on dist-test-slave-8w9v"
server_key: "87021120296bdeb4d088d0557cbcbe45"
server_key_iv: "67ed553ed1d923b41cb2c2736d5bfef2"
server_key_version: "encryptionkey@0"
I20260809 15:18:40.982925 5549 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 15:18:40.986311 5565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:40.987596 5549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:18:40.987802 5549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "1ff3981347d741bfb39b9b89fc9b1769"
format_stamp: "Formatted at 2026-08-09 15:18:40 on dist-test-slave-8w9v"
server_key: "87021120296bdeb4d088d0557cbcbe45"
server_key_iv: "67ed553ed1d923b41cb2c2736d5bfef2"
server_key_version: "encryptionkey@0"
I20260809 15:18:40.988009 5549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:40.993710 5549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:40.997991 5549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:40.998288 5549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:41.017355 5549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45119
I20260809 15:18:41.017411 5618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45119 every 8 connection(s)
I20260809 15:18:41.018761 5549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:41.021039 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 5549
I20260809 15:18:41.021245 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:41.021821 15406 external_mini_cluster.cc:1428] Setting key ad283b0a0341f49efaa2fa7f5696946f
I20260809 15:18:41.023077 5619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:41.029546 5622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:41.023624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54739 (local address 127.0.0.1:45119)
0809 15:18:41.024442 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:41.024453 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:41.024495 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:18:41.026105 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:41.026125 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:41.026466 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:41.026889 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:41.027922 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:41.027941 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:41.027953 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:41.027968 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:41.028042 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:41.028067 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:41.028323 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:41.028470 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 15:18:41.028646 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":367,"threads_started":1}
I20260809 15:18:41.033005 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:41.035140 5619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769: Bootstrap starting.
I20260809 15:18:41.038647 5619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:41.039819 5622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:41.037314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58791 (local address 127.0.0.1:45119)
0809 15:18:41.037470 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:41.037474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:41.037901 (+ 427us) server_negotiation.cc:407] Connection header received
0809 15:18:41.038088 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:41.038093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:41.038134 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:41.038229 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:41.039148 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:41.039153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:41.039158 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:41.039175 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:41.039217 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:41.039231 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:41.039344 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:41.039491 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 15:18:41.039550 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:18:41.039824 5619 log.cc:824] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:41.042824 5619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769: No bootstrap required, opened a new log
I20260809 15:18:41.050225 5619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff3981347d741bfb39b9b89fc9b1769" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } }
I20260809 15:18:41.050559 5619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:41.050679 5619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ff3981347d741bfb39b9b89fc9b1769, State: Initialized, Role: FOLLOWER
I20260809 15:18:41.051244 5619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff3981347d741bfb39b9b89fc9b1769" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } }
I20260809 15:18:41.051474 5619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:41.051637 5619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:41.051812 5619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:41.053957 5619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff3981347d741bfb39b9b89fc9b1769" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } }
I20260809 15:18:41.054443 5619 leader_election.cc:302] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ff3981347d741bfb39b9b89fc9b1769; no voters:
I20260809 15:18:41.055069 5619 leader_election.cc:288] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:41.055220 5632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:41.060391 5632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [term 1 LEADER]: Becoming Leader. State: Replica: 1ff3981347d741bfb39b9b89fc9b1769, State: Running, Role: LEADER
I20260809 15:18:41.060989 5632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff3981347d741bfb39b9b89fc9b1769" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } }
I20260809 15:18:41.061863 5619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:41.063845 5635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ff3981347d741bfb39b9b89fc9b1769. Latest consensus state: current_term: 1 leader_uuid: "1ff3981347d741bfb39b9b89fc9b1769" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff3981347d741bfb39b9b89fc9b1769" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } } }
I20260809 15:18:41.064309 5635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:41.064266 5633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ff3981347d741bfb39b9b89fc9b1769" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff3981347d741bfb39b9b89fc9b1769" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45119 } } }
I20260809 15:18:41.065240 5633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:41.066125 5638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:41.072297 5638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:41.081020 5638 catalog_manager.cc:1347] Generated new cluster ID: 801cc46ce7764484a8ba4201c2d75467
I20260809 15:18:41.081125 5638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:41.098064 5638 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:41.098454 5638 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:41.104844 5638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ff3981347d741bfb39b9b89fc9b1769: Generated new TSK 0
I20260809 15:18:41.105736 5638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:41.115185 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5549
W20260809 15:18:41.116456 5614 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54739
W20260809 15:18:41.117264 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5549: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:18:41.117427 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 5549
2026-08-09T15:18:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1958 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:41 dist-test-slave-8w9v krb5kdc[5656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:41 dist-test-slave-8w9v krb5kdc[5656](info): set up 2 sockets
Aug 09 15:18:41 dist-test-slave-8w9v krb5kdc[5656](info): commencing operation
krb5kdc: starting...
W20260809 15:18:42.270897 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:42 dist-test-slave-8w9v krb5kdc[5656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:42.583446 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46663
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:42.918252 5670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:42.918540 5670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:42.918753 5670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:42.925832 5670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:42.925931 5670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:42.925979 5670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:42.926093 5670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:42.926182 5670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:42.935690 5670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:42.937016 5670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:42.938318 5670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:42.949509 5675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:42.949523 5676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:42.949717 5678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:42.950570 5670 server_base.cc:1034] running on GCE node
I20260809 15:18:42.951292 5670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:42.952687 5670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:42.953940 5670 hybrid_clock.cc:648] HybridClock initialized: now 1786288722953909 us; error 57 us; skew 500 ppm
I20260809 15:18:42.954322 5670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:42 dist-test-slave-8w9v krb5kdc[5656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288722, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:42.957438 5670 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:42.958913 5670 webserver.cc:460] Webserver started at http://127.0.0.1:45825/ using document root <none> and password file <none>
I20260809 15:18:42.959591 5670 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:42.959750 5670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:42.960104 5670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:42.963173 5670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "51625bda25da48998f3103012c00c446"
format_stamp: "Formatted at 2026-08-09 15:18:42 on dist-test-slave-8w9v"
server_key: "835331534ab39e05778e0594b3df6e8a"
server_key_iv: "12a0d6a257d7c4b24d62255612280f55"
server_key_version: "encryptionkey@0"
I20260809 15:18:42.964001 5670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "51625bda25da48998f3103012c00c446"
format_stamp: "Formatted at 2026-08-09 15:18:42 on dist-test-slave-8w9v"
server_key: "835331534ab39e05778e0594b3df6e8a"
server_key_iv: "12a0d6a257d7c4b24d62255612280f55"
server_key_version: "encryptionkey@0"
I20260809 15:18:42.969122 5670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:18:42.973032 5686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:42.974258 5670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:18:42.974452 5670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "51625bda25da48998f3103012c00c446"
format_stamp: "Formatted at 2026-08-09 15:18:42 on dist-test-slave-8w9v"
server_key: "835331534ab39e05778e0594b3df6e8a"
server_key_iv: "12a0d6a257d7c4b24d62255612280f55"
server_key_version: "encryptionkey@0"
I20260809 15:18:42.974640 5670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:43.005244 5670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:43.009232 5670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:43.009539 5670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:43.028646 5670 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46663
I20260809 15:18:43.028664 5743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46663 every 8 connection(s)
I20260809 15:18:43.030165 5670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:43.033310 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 5670
I20260809 15:18:43.033492 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:43.034060 15406 external_mini_cluster.cc:1428] Setting key a9791b796099b42f5da42fbe99f544a0
I20260809 15:18:43.034408 5744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:43.046499 5744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446: Bootstrap starting.
Aug 09 15:18:43 dist-test-slave-8w9v krb5kdc[5656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:18:43.049860 5744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:43.051095 5744 log.cc:824] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:43.053962 5744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446: No bootstrap required, opened a new log
I20260809 15:18:43.057058 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:43.035899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39813 (local address 127.0.0.1:46663)
0809 15:18:43.036758 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:43.036770 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:43.036823 (+ 53us) server_negotiation.cc:407] Connection header received
0809 15:18:43.039020 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:43.039042 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:43.039419 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:43.039815 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:43.041306 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:43.042556 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:43.043856 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:43.044624 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:43.044833 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:43.049155 (+ 4322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:43.049162 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:43.049176 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:43.049215 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:43.052911 (+ 3696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:43.053589 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:43.053595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:43.053601 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:43.053757 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:43.054185 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:43.054192 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:43.054195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:43.054745 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:43.055053 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:43.055611 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 15:18:43.056024 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":392,"threads_started":1}
I20260809 15:18:43.060775 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:43.062463 5744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51625bda25da48998f3103012c00c446" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } }
I20260809 15:18:43.062829 5744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:43.062948 5744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51625bda25da48998f3103012c00c446, State: Initialized, Role: FOLLOWER
I20260809 15:18:43.063539 5744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51625bda25da48998f3103012c00c446" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } }
I20260809 15:18:43.063781 5744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:43.063937 5744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:43.064165 5744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:43.066540 5744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51625bda25da48998f3103012c00c446" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } }
I20260809 15:18:43.067006 5744 leader_election.cc:302] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51625bda25da48998f3103012c00c446; no voters:
I20260809 15:18:43.067651 5744 leader_election.cc:288] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:43.067739 5754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:43.070888 5744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:43.072417 5754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [term 1 LEADER]: Becoming Leader. State: Replica: 51625bda25da48998f3103012c00c446, State: Running, Role: LEADER
I20260809 15:18:43.073166 5754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51625bda25da48998f3103012c00c446" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } }
I20260809 15:18:43.077044 5755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51625bda25da48998f3103012c00c446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51625bda25da48998f3103012c00c446" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } } }
I20260809 15:18:43.077579 5755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:43.079512 5761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:43.079263 5756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51625bda25da48998f3103012c00c446. Latest consensus state: current_term: 1 leader_uuid: "51625bda25da48998f3103012c00c446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51625bda25da48998f3103012c00c446" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46663 } } }
I20260809 15:18:43.081091 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:43.065184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49477 (local address 127.0.0.1:46663)
0809 15:18:43.065416 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:43.065429 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:43.065963 (+ 534us) server_negotiation.cc:407] Connection header received
0809 15:18:43.066155 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:43.066160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:43.066225 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:43.066320 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:43.067945 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:43.069120 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:43.070470 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:43.070937 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:43.071092 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:43.073865 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:43.073871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:43.073874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:43.073903 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:43.077716 (+ 3813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:43.078623 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:43.078630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:43.078635 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:43.078801 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:43.079306 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:43.079310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:43.079313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:43.079779 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:43.080033 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:43.080501 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 15:18:43.080706 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:18:43.080312 5756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:43.087833 5761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:43.097638 5761 catalog_manager.cc:1347] Generated new cluster ID: b6f2d76ad9064f3698c742f5634685ef
I20260809 15:18:43.097734 5761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:43.137506 5761 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:43.139511 5761 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:43.149056 5761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51625bda25da48998f3103012c00c446: Generated new TSK 0
I20260809 15:18:43.149847 5761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:43.155342 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5670
I20260809 15:18:43.156487 5739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:43.246562 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 5670
2026-08-09T15:18:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2129 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:43 dist-test-slave-8w9v krb5kdc[5778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:43 dist-test-slave-8w9v krb5kdc[5778](info): set up 2 sockets
Aug 09 15:18:43 dist-test-slave-8w9v krb5kdc[5778](info): commencing operation
krb5kdc: starting...
W20260809 15:18:44.428287 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.002s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:44 dist-test-slave-8w9v krb5kdc[5778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:44.714883 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40291
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:45.053066 5791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:45.053332 5791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:45.053508 5791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:45.060248 5791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:45.060365 5791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:45.060460 5791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:45.060562 5791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:45.060636 5791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:45.069936 5791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:45.071159 5791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:45.072530 5791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:45.084090 5797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:45.084138 5798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:45.084162 5800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:45.085299 5791 server_base.cc:1034] running on GCE node
I20260809 15:18:45.086048 5791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:45.087390 5791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:45.088618 5791 hybrid_clock.cc:648] HybridClock initialized: now 1786288725088532 us; error 76 us; skew 500 ppm
I20260809 15:18:45.088944 5791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:45 dist-test-slave-8w9v krb5kdc[5778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288725, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:45.091960 5791 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:45.093385 5791 webserver.cc:460] Webserver started at http://127.0.0.1:45385/ using document root <none> and password file <none>
I20260809 15:18:45.094060 5791 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:45.094216 5791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:45.094516 5791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:45.097506 5791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "a3e02433ebaf481eb6da703a5ed5dad7"
format_stamp: "Formatted at 2026-08-09 15:18:45 on dist-test-slave-8w9v"
server_key: "cf2aa771499cb92f99c84626ec0b3961"
server_key_iv: "7351f8c9e78ffa46804e470c1759f914"
server_key_version: "encryptionkey@0"
I20260809 15:18:45.098246 5791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "a3e02433ebaf481eb6da703a5ed5dad7"
format_stamp: "Formatted at 2026-08-09 15:18:45 on dist-test-slave-8w9v"
server_key: "cf2aa771499cb92f99c84626ec0b3961"
server_key_iv: "7351f8c9e78ffa46804e470c1759f914"
server_key_version: "encryptionkey@0"
I20260809 15:18:45.103096 5791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:18:45.106258 5807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:45.107431 5791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:18:45.107607 5791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "a3e02433ebaf481eb6da703a5ed5dad7"
format_stamp: "Formatted at 2026-08-09 15:18:45 on dist-test-slave-8w9v"
server_key: "cf2aa771499cb92f99c84626ec0b3961"
server_key_iv: "7351f8c9e78ffa46804e470c1759f914"
server_key_version: "encryptionkey@0"
I20260809 15:18:45.107788 5791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:45.122370 5791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:45.126389 5791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:45.126667 5791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:45.145310 5791 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40291
I20260809 15:18:45.145349 5866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40291 every 8 connection(s)
I20260809 15:18:45.146663 5791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:45.150219 5867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:45.152861 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 5791
I20260809 15:18:45.153045 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:45.153488 15406 external_mini_cluster.cc:1428] Setting key e5008d5b63b69305b3e26c0cc621134b
I20260809 15:18:45.163203 5867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7: Bootstrap starting.
I20260809 15:18:45.167273 5867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:45.168588 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:45.155360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57269 (local address 127.0.0.1:40291)
0809 15:18:45.156200 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:45.156215 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:45.156262 (+ 47us) server_negotiation.cc:407] Connection header received
0809 15:18:45.158547 (+ 2285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:45.158575 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:45.159090 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:45.159505 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:45.160040 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:45.162081 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:45.163555 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:45.164579 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:45.164776 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:45.165838 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:45.165847 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:45.165862 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:45.166574 (+ 712us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:45.166668 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:45.166699 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:45.166801 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:45.166983 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 15:18:45.167246 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":322,"threads_started":1}
I20260809 15:18:45.168640 5867 log.cc:824] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:45.171950 5867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7: No bootstrap required, opened a new log
I20260809 15:18:45.172621 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:45.180253 5867 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e02433ebaf481eb6da703a5ed5dad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40291 } }
I20260809 15:18:45.180596 5867 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:45.180722 5867 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e02433ebaf481eb6da703a5ed5dad7, State: Initialized, Role: FOLLOWER
I20260809 15:18:45.181277 5867 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e02433ebaf481eb6da703a5ed5dad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40291 } }
I20260809 15:18:45.181546 5867 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:45.181686 5867 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:45.181887 5867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:45.183528 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:45.177092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56091 (local address 127.0.0.1:40291)
0809 15:18:45.177330 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:45.177338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:45.177876 (+ 538us) server_negotiation.cc:407] Connection header received
0809 15:18:45.178014 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:45.178024 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:45.178100 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:45.178204 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:45.178638 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:45.179702 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:45.181109 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:45.181588 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:45.181730 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:45.182822 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:45.182827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:45.182830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:45.182848 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:45.182884 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:45.182901 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:45.183008 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:45.183150 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 15:18:45.183244 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:18:45.184553 5867 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e02433ebaf481eb6da703a5ed5dad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40291 } }
I20260809 15:18:45.185055 5867 leader_election.cc:302] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3e02433ebaf481eb6da703a5ed5dad7; no voters:
I20260809 15:18:45.185660 5867 leader_election.cc:288] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:45.185756 5877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:45.186599 5877 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [term 1 LEADER]: Becoming Leader. State: Replica: a3e02433ebaf481eb6da703a5ed5dad7, State: Running, Role: LEADER
I20260809 15:18:45.187206 5877 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e02433ebaf481eb6da703a5ed5dad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40291 } }
I20260809 15:18:45.188113 5867 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:45.190982 5879 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3e02433ebaf481eb6da703a5ed5dad7. Latest consensus state: current_term: 1 leader_uuid: "a3e02433ebaf481eb6da703a5ed5dad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e02433ebaf481eb6da703a5ed5dad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40291 } } }
I20260809 15:18:45.191293 5878 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3e02433ebaf481eb6da703a5ed5dad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3e02433ebaf481eb6da703a5ed5dad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40291 } } }
I20260809 15:18:45.191501 5879 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:45.191572 5878 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:45.192710 5884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:45.198980 5884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:45.209563 5884 catalog_manager.cc:1347] Generated new cluster ID: db36915463e147daad7c8e39ecd77500
I20260809 15:18:45.209692 5884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:45.221835 5884 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:45.223150 5884 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:45.230466 5884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3e02433ebaf481eb6da703a5ed5dad7: Generated new TSK 0
I20260809 15:18:45.231192 5884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:45.240682 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5791
W20260809 15:18:45.241823 5862 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57269
W20260809 15:18:45.242540 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5791: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:18:45.242686 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 5791
2026-08-09T15:18:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1995 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:45 dist-test-slave-8w9v krb5kdc[5898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:45 dist-test-slave-8w9v krb5kdc[5898](info): set up 2 sockets
Aug 09 15:18:45 dist-test-slave-8w9v krb5kdc[5898](info): commencing operation
krb5kdc: starting...
W20260809 15:18:46.384992 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:46 dist-test-slave-8w9v krb5kdc[5898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:46.680437 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44993
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:47.023114 5911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:47.023461 5911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:47.023610 5911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:47.031087 5911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:47.031205 5911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:47.031252 5911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:47.031356 5911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:47.031446 5911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:47.041463 5911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:47.042841 5911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:47.044230 5911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:47.056394 5916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:47.056340 5917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:47.056903 5919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:47.057325 5911 server_base.cc:1034] running on GCE node
I20260809 15:18:47.058048 5911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:47.059448 5911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:47.060654 5911 hybrid_clock.cc:648] HybridClock initialized: now 1786288727060597 us; error 51 us; skew 500 ppm
I20260809 15:18:47.061020 5911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:47 dist-test-slave-8w9v krb5kdc[5898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288727, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:47.069278 5911 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:47.070932 5911 webserver.cc:460] Webserver started at http://127.0.0.1:42899/ using document root <none> and password file <none>
I20260809 15:18:47.071671 5911 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:47.071805 5911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:47.072196 5911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:47.075269 5911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93"
format_stamp: "Formatted at 2026-08-09 15:18:47 on dist-test-slave-8w9v"
server_key: "d0941a3ac3e076a0d5a4c504a3d1a0c5"
server_key_iv: "de0655d0257b5ddb237d5b419d278392"
server_key_version: "encryptionkey@0"
I20260809 15:18:47.076098 5911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93"
format_stamp: "Formatted at 2026-08-09 15:18:47 on dist-test-slave-8w9v"
server_key: "d0941a3ac3e076a0d5a4c504a3d1a0c5"
server_key_iv: "de0655d0257b5ddb237d5b419d278392"
server_key_version: "encryptionkey@0"
I20260809 15:18:47.081511 5911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 15:18:47.085019 5927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:47.086410 5911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:18:47.086603 5911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93"
format_stamp: "Formatted at 2026-08-09 15:18:47 on dist-test-slave-8w9v"
server_key: "d0941a3ac3e076a0d5a4c504a3d1a0c5"
server_key_iv: "de0655d0257b5ddb237d5b419d278392"
server_key_version: "encryptionkey@0"
I20260809 15:18:47.086807 5911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:47.092944 5911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:47.097383 5911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:47.097681 5911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:47.116472 5911 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44993
I20260809 15:18:47.116554 5980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44993 every 8 connection(s)
I20260809 15:18:47.117847 5911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:47.120622 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 5911
I20260809 15:18:47.120788 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:47.121276 15406 external_mini_cluster.cc:1428] Setting key fabe3010e9ca5c8aff8eef2e89fb8aef
I20260809 15:18:47.121637 5981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:47.129513 5984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:47.123083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59073 (local address 127.0.0.1:44993)
0809 15:18:47.123914 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:47.123928 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:47.123969 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:18:47.126190 (+ 2221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:47.126209 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:47.126536 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:47.126858 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:47.127935 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:47.127941 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:47.127956 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:47.127969 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:47.128032 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:47.128050 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:47.128299 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:47.128421 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 15:18:47.128618 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":372,"threads_started":1}
I20260809 15:18:47.132992 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:47.135246 5981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93: Bootstrap starting.
I20260809 15:18:47.139211 5981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:47.140426 5981 log.cc:824] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:47.140668 5984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:47.138105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57063 (local address 127.0.0.1:44993)
0809 15:18:47.138280 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:47.138286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:47.138749 (+ 463us) server_negotiation.cc:407] Connection header received
0809 15:18:47.138869 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:47.138873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:47.138928 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:47.139029 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:47.139943 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:47.139951 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:47.139958 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:47.139975 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:47.140022 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:47.140049 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:47.140199 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:47.140282 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 15:18:47.140333 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:18:47.143220 5981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93: No bootstrap required, opened a new log
I20260809 15:18:47.150218 5981 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } }
I20260809 15:18:47.150520 5981 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:47.150597 5981 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdeb6bcffe774c1db7ac8e13ff2cae93, State: Initialized, Role: FOLLOWER
I20260809 15:18:47.151086 5981 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } }
I20260809 15:18:47.151278 5981 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:47.151372 5981 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:47.151526 5981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:47.153877 5981 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } }
I20260809 15:18:47.154356 5981 leader_election.cc:302] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdeb6bcffe774c1db7ac8e13ff2cae93; no voters:
I20260809 15:18:47.154956 5981 leader_election.cc:288] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:47.155102 5993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:47.156598 5993 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [term 1 LEADER]: Becoming Leader. State: Replica: fdeb6bcffe774c1db7ac8e13ff2cae93, State: Running, Role: LEADER
I20260809 15:18:47.157078 5993 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } }
I20260809 15:18:47.157218 5981 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:47.159557 5995 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdeb6bcffe774c1db7ac8e13ff2cae93. Latest consensus state: current_term: 1 leader_uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } } }
I20260809 15:18:47.159885 5995 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:47.160164 5994 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdeb6bcffe774c1db7ac8e13ff2cae93" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44993 } } }
I20260809 15:18:47.160480 5994 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:47.161877 6000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:47.167145 6000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:47.175719 6000 catalog_manager.cc:1347] Generated new cluster ID: af2c03aac4fa407d9723a62f537a2cb6
I20260809 15:18:47.175897 6000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:47.191942 6000 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:47.192404 6000 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:47.199397 6000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdeb6bcffe774c1db7ac8e13ff2cae93: Generated new TSK 0
I20260809 15:18:47.200114 6000 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:47.208600 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5911
W20260809 15:18:47.209659 5976 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59073
W20260809 15:18:47.210438 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 5911: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:18:47.210598 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 5911
2026-08-09T15:18:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1967 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:47 dist-test-slave-8w9v krb5kdc[6017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:47 dist-test-slave-8w9v krb5kdc[6017](info): set up 2 sockets
Aug 09 15:18:47 dist-test-slave-8w9v krb5kdc[6017](info): commencing operation
krb5kdc: starting...
W20260809 15:18:48.367767 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.005s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:48 dist-test-slave-8w9v krb5kdc[6017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:48.675278 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:49.026880 6031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:49.027202 6031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:49.027396 6031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:49.034731 6031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:49.034860 6031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:49.034953 6031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:49.035054 6031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:49.035130 6031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:49.044981 6031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:49.046307 6031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:49.047595 6031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:49.059262 6037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:49.059222 6038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:49.059185 6040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:49.060068 6031 server_base.cc:1034] running on GCE node
I20260809 15:18:49.060822 6031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:49.062189 6031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:49.063424 6031 hybrid_clock.cc:648] HybridClock initialized: now 1786288729063389 us; error 61 us; skew 500 ppm
I20260809 15:18:49.063776 6031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:49 dist-test-slave-8w9v krb5kdc[6017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288729, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:49.066823 6031 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:49.068270 6031 webserver.cc:460] Webserver started at http://127.0.0.1:44305/ using document root <none> and password file <none>
I20260809 15:18:49.068919 6031 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:49.069067 6031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:49.069375 6031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:49.072299 6031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "2e9591984a334d868d7c58b0e515aa05"
format_stamp: "Formatted at 2026-08-09 15:18:49 on dist-test-slave-8w9v"
server_key: "14c78fae0aaefcd1d8fd1b9ad13c0656"
server_key_iv: "b3fb63d837954d1af808f4f2b8610f26"
server_key_version: "encryptionkey@0"
I20260809 15:18:49.073102 6031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "2e9591984a334d868d7c58b0e515aa05"
format_stamp: "Formatted at 2026-08-09 15:18:49 on dist-test-slave-8w9v"
server_key: "14c78fae0aaefcd1d8fd1b9ad13c0656"
server_key_iv: "b3fb63d837954d1af808f4f2b8610f26"
server_key_version: "encryptionkey@0"
I20260809 15:18:49.077927 6031 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:18:49.081152 6048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:49.082432 6031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:18:49.082624 6031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "2e9591984a334d868d7c58b0e515aa05"
format_stamp: "Formatted at 2026-08-09 15:18:49 on dist-test-slave-8w9v"
server_key: "14c78fae0aaefcd1d8fd1b9ad13c0656"
server_key_iv: "b3fb63d837954d1af808f4f2b8610f26"
server_key_version: "encryptionkey@0"
I20260809 15:18:49.082813 6031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:49.105089 6031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:49.109086 6031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:49.109359 6031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:49.127383 6031 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41477
I20260809 15:18:49.127463 6107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41477 every 8 connection(s)
I20260809 15:18:49.128803 6031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:49.132390 6109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:49.135532 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 6031
I20260809 15:18:49.135720 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:49.136277 15406 external_mini_cluster.cc:1428] Setting key 3eeda5842084d6fbf2d731b0fb162c7c
I20260809 15:18:49.144775 6109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05: Bootstrap starting.
I20260809 15:18:49.148656 6109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:49.150044 6109 log.cc:824] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:18:49 dist-test-slave-8w9v krb5kdc[6017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:18:49.153398 6109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05: No bootstrap required, opened a new log
I20260809 15:18:49.160282 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:49.138121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:40467 (local address 127.0.0.1:41477)
0809 15:18:49.138883 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:49.138895 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:49.138935 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:18:49.140590 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:49.140611 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:49.140998 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:49.141322 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:49.143137 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:49.144531 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:49.146340 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:49.147038 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:49.147162 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:49.152666 (+ 5504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:49.152674 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:49.152687 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:49.152720 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:49.155984 (+ 3264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:49.156937 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:49.156943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:49.156948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:49.157109 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:49.157591 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:49.157594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:49.157596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:49.158102 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:49.158362 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:49.158844 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 15:18:49.159127 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":350,"threads_started":1}
I20260809 15:18:49.162298 6109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e9591984a334d868d7c58b0e515aa05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41477 } }
I20260809 15:18:49.162664 6109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:49.162783 6109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e9591984a334d868d7c58b0e515aa05, State: Initialized, Role: FOLLOWER
I20260809 15:18:49.163331 6109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e9591984a334d868d7c58b0e515aa05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41477 } }
I20260809 15:18:49.163578 6109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:49.163738 6109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:49.163934 6109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:49.164193 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:49.167014 6109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e9591984a334d868d7c58b0e515aa05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41477 } }
I20260809 15:18:49.167603 6109 leader_election.cc:302] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e9591984a334d868d7c58b0e515aa05; no voters:
I20260809 15:18:49.168435 6109 leader_election.cc:288] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:49.168524 6118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:49.170665 6118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [term 1 LEADER]: Becoming Leader. State: Replica: 2e9591984a334d868d7c58b0e515aa05, State: Running, Role: LEADER
I20260809 15:18:49.170746 6109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:49.171331 6118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e9591984a334d868d7c58b0e515aa05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41477 } }
I20260809 15:18:49.175275 6121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e9591984a334d868d7c58b0e515aa05. Latest consensus state: current_term: 1 leader_uuid: "2e9591984a334d868d7c58b0e515aa05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e9591984a334d868d7c58b0e515aa05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41477 } } }
I20260809 15:18:49.175655 6121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:49.176290 6120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e9591984a334d868d7c58b0e515aa05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e9591984a334d868d7c58b0e515aa05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41477 } } }
I20260809 15:18:49.176687 6120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:49.186798 6131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:49.190217 6113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:49.170048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:59635 (local address 127.0.0.1:41477)
0809 15:18:49.170216 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:49.170221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:49.170295 (+ 74us) server_negotiation.cc:407] Connection header received
0809 15:18:49.170787 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:49.170794 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:49.170875 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:49.170946 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:49.173232 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:49.174997 (+ 1765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:49.177431 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:49.178351 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:49.178483 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:49.182427 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:49.182436 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:49.182444 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:49.182493 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:49.187091 (+ 4598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:49.187893 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:49.187898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:49.187900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:49.188060 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:49.188548 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:49.188553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:49.188557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:49.188952 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:49.189261 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:49.189662 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 15:18:49.189838 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:18:49.192040 6131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:49.199749 6131 catalog_manager.cc:1347] Generated new cluster ID: 12db837fdf7d4b63ac897b2a48dea4ee
I20260809 15:18:49.199856 6131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:49.223948 6131 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:49.225793 6131 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:49.246452 6131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e9591984a334d868d7c58b0e515aa05: Generated new TSK 0
I20260809 15:18:49.247486 6131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:49.257246 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6031
I20260809 15:18:49.258875 6103 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:49.347692 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 6031
W20260809 15:18:49.368611 6015 connection.cc:535] client connection to 127.0.0.1:41477 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41477): Connection reset by peer (error 104)
2026-08-09T15:18:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2136 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:49 dist-test-slave-8w9v krb5kdc[6144](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:49 dist-test-slave-8w9v krb5kdc[6144](info): set up 2 sockets
Aug 09 15:18:49 dist-test-slave-8w9v krb5kdc[6144](info): commencing operation
krb5kdc: starting...
W20260809 15:18:50.492655 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:50 dist-test-slave-8w9v krb5kdc[6144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:50.789690 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:51.129300 6159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:51.129570 6159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:51.129756 6159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:51.136771 6159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:51.136889 6159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:51.136981 6159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:51.137092 6159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:51.137164 6159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:51.146772 6159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:51.148013 6159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:51.149366 6159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:51.161065 6165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:51.161151 6169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:51.161113 6166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:51.162274 6159 server_base.cc:1034] running on GCE node
I20260809 15:18:51.163035 6159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:51.164398 6159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:51.165593 6159 hybrid_clock.cc:648] HybridClock initialized: now 1786288731165538 us; error 51 us; skew 500 ppm
I20260809 15:18:51.165923 6159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:51 dist-test-slave-8w9v krb5kdc[6144](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288731, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:51.169133 6159 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:51.170478 6159 webserver.cc:460] Webserver started at http://127.0.0.1:46175/ using document root <none> and password file <none>
I20260809 15:18:51.171056 6159 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:51.171152 6159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:51.171401 6159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:51.174209 6159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "0b200a90594e45df8d0820ec6d91411b"
format_stamp: "Formatted at 2026-08-09 15:18:51 on dist-test-slave-8w9v"
server_key: "0971b9ebe35d2ed42f67eec59a8a5801"
server_key_iv: "42c75e59d2b6be0441aa1e081f4bfd90"
server_key_version: "encryptionkey@0"
I20260809 15:18:51.174904 6159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "0b200a90594e45df8d0820ec6d91411b"
format_stamp: "Formatted at 2026-08-09 15:18:51 on dist-test-slave-8w9v"
server_key: "0971b9ebe35d2ed42f67eec59a8a5801"
server_key_iv: "42c75e59d2b6be0441aa1e081f4bfd90"
server_key_version: "encryptionkey@0"
I20260809 15:18:51.179853 6159 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 15:18:51.183225 6177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:51.184561 6159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:18:51.184739 6159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "0b200a90594e45df8d0820ec6d91411b"
format_stamp: "Formatted at 2026-08-09 15:18:51 on dist-test-slave-8w9v"
server_key: "0971b9ebe35d2ed42f67eec59a8a5801"
server_key_iv: "42c75e59d2b6be0441aa1e081f4bfd90"
server_key_version: "encryptionkey@0"
I20260809 15:18:51.184923 6159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:51.210500 6159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:51.214881 6159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:51.215157 6159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:51.233441 6159 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41051
I20260809 15:18:51.233477 6230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41051 every 8 connection(s)
I20260809 15:18:51.234807 6159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:51.237932 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 6159
I20260809 15:18:51.238121 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:51.238596 15406 external_mini_cluster.cc:1428] Setting key 235b93c1c97704fe054dc4efb0a0722b
I20260809 15:18:51.238648 6231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:51.253720 6231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b: Bootstrap starting.
I20260809 15:18:51.253937 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:51.240512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:43615 (local address 127.0.0.1:41051)
0809 15:18:51.241298 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:51.241313 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:51.241361 (+ 48us) server_negotiation.cc:407] Connection header received
0809 15:18:51.243516 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:51.243544 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:51.244075 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:51.244529 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:51.245069 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:51.247083 (+ 2014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:51.248562 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:51.249635 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:51.249821 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:51.251164 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:51.251175 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:51.251191 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:51.251902 (+ 711us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:51.251993 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:51.252023 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:51.252208 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:51.252467 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 15:18:51.252647 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":303,"threads_started":1}
I20260809 15:18:51.257283 6231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:51.257856 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:51.258430 6231 log.cc:824] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:51.261340 6231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b: No bootstrap required, opened a new log
I20260809 15:18:51.269512 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:51.262718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:54723 (local address 127.0.0.1:41051)
0809 15:18:51.262955 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:51.262963 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:51.263450 (+ 487us) server_negotiation.cc:407] Connection header received
0809 15:18:51.263637 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:51.263641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:51.263701 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:51.263788 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:51.264388 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:51.265121 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:51.266997 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:51.267448 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:51.267553 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:51.268715 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:51.268719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:51.268722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:51.268737 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:51.268774 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:51.268787 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:51.268886 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:51.269088 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 15:18:51.269128 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:18:51.270897 6231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b200a90594e45df8d0820ec6d91411b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } }
I20260809 15:18:51.271291 6231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:51.271409 6231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b200a90594e45df8d0820ec6d91411b, State: Initialized, Role: FOLLOWER
I20260809 15:18:51.271965 6231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b200a90594e45df8d0820ec6d91411b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } }
I20260809 15:18:51.272295 6231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:51.272485 6231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:51.272675 6231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:51.274904 6231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b200a90594e45df8d0820ec6d91411b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } }
I20260809 15:18:51.275372 6231 leader_election.cc:302] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b200a90594e45df8d0820ec6d91411b; no voters:
I20260809 15:18:51.275944 6231 leader_election.cc:288] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:51.276065 6241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:51.276918 6241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [term 1 LEADER]: Becoming Leader. State: Replica: 0b200a90594e45df8d0820ec6d91411b, State: Running, Role: LEADER
I20260809 15:18:51.277613 6241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b200a90594e45df8d0820ec6d91411b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } }
I20260809 15:18:51.278424 6231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:51.280488 6243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b200a90594e45df8d0820ec6d91411b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b200a90594e45df8d0820ec6d91411b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } } }
I20260809 15:18:51.280512 6244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b200a90594e45df8d0820ec6d91411b. Latest consensus state: current_term: 1 leader_uuid: "0b200a90594e45df8d0820ec6d91411b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b200a90594e45df8d0820ec6d91411b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41051 } } }
I20260809 15:18:51.280934 6243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:51.280934 6244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:51.284071 6249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:51.288736 6249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:51.297551 6249 catalog_manager.cc:1347] Generated new cluster ID: a8a568c76fbc4ea983eebf72bd20ae5f
I20260809 15:18:51.297662 6249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:51.305756 6249 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:51.306967 6249 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:51.315604 6249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b200a90594e45df8d0820ec6d91411b: Generated new TSK 0
I20260809 15:18:51.316473 6249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:51.326799 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6159
W20260809 15:18:51.328002 6226 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.214:43615
W20260809 15:18:51.328790 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6159: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:18:51.328944 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 6159
W20260809 15:18:51.351153 6142 connection.cc:535] client connection to 127.0.0.1:41051 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41051): Connection reset by peer (error 104)
2026-08-09T15:18:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1982 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:51 dist-test-slave-8w9v krb5kdc[6266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:51 dist-test-slave-8w9v krb5kdc[6266](info): set up 2 sockets
Aug 09 15:18:51 dist-test-slave-8w9v krb5kdc[6266](info): commencing operation
krb5kdc: starting...
W20260809 15:18:52.472815 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:52 dist-test-slave-8w9v krb5kdc[6266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:52.773815 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:53.115388 6279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:53.115651 6279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:53.115803 6279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:53.122856 6279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:53.122954 6279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:53.123090 6279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:53.123188 6279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:53.123270 6279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:53.132854 6279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:53.134086 6279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:53.135447 6279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:53.147811 6284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:53.147750 6287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:53.147857 6285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:53.148849 6279 server_base.cc:1034] running on GCE node
I20260809 15:18:53.149571 6279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:53.150893 6279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:53.152107 6279 hybrid_clock.cc:648] HybridClock initialized: now 1786288733152066 us; error 30 us; skew 500 ppm
I20260809 15:18:53.152465 6279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:53 dist-test-slave-8w9v krb5kdc[6266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288733, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:53.159405 6279 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:53.161028 6279 webserver.cc:460] Webserver started at http://127.0.0.1:42819/ using document root <none> and password file <none>
I20260809 15:18:53.161746 6279 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:53.161898 6279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:53.162218 6279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:53.165354 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "37ea66ad862e428aaacfd118d12be21f"
format_stamp: "Formatted at 2026-08-09 15:18:53 on dist-test-slave-8w9v"
server_key: "832e6e99bd92dd7621cc5a230443516f"
server_key_iv: "9ada14dccb12c9db25abdfa663d35cbf"
server_key_version: "encryptionkey@0"
I20260809 15:18:53.166172 6279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "37ea66ad862e428aaacfd118d12be21f"
format_stamp: "Formatted at 2026-08-09 15:18:53 on dist-test-slave-8w9v"
server_key: "832e6e99bd92dd7621cc5a230443516f"
server_key_iv: "9ada14dccb12c9db25abdfa663d35cbf"
server_key_version: "encryptionkey@0"
I20260809 15:18:53.171336 6279 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 15:18:53.174805 6295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:53.176222 6279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:18:53.176401 6279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "37ea66ad862e428aaacfd118d12be21f"
format_stamp: "Formatted at 2026-08-09 15:18:53 on dist-test-slave-8w9v"
server_key: "832e6e99bd92dd7621cc5a230443516f"
server_key_iv: "9ada14dccb12c9db25abdfa663d35cbf"
server_key_version: "encryptionkey@0"
I20260809 15:18:53.176595 6279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:53.182317 6279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:53.186281 6279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:53.186591 6279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:53.205439 6279 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44567
I20260809 15:18:53.205518 6350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44567 every 8 connection(s)
I20260809 15:18:53.206785 6279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:53.210831 6351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:53.213433 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 6279
I20260809 15:18:53.213617 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:53.214035 15406 external_mini_cluster.cc:1428] Setting key a90444b397b8f75c0be670092e697b45
I20260809 15:18:53.223383 6354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:53.215833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:46261 (local address 127.0.0.1:44567)
0809 15:18:53.216755 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:53.216775 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:53.216834 (+ 59us) server_negotiation.cc:407] Connection header received
0809 15:18:53.219263 (+ 2429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:53.219289 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:53.219727 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:53.220214 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:53.221242 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:53.221251 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:53.221269 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:53.221288 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:53.221369 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:53.221421 (+ 52us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:53.221681 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:53.221858 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 15:18:53.222079 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":379,"threads_started":1}
I20260809 15:18:53.225210 6351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f: Bootstrap starting.
I20260809 15:18:53.226946 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:53.228758 6351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:53.229930 6351 log.cc:824] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:53.233144 6351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f: No bootstrap required, opened a new log
I20260809 15:18:53.234110 6354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:53.231502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:42439 (local address 127.0.0.1:44567)
0809 15:18:53.231685 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:53.231691 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:53.232140 (+ 449us) server_negotiation.cc:407] Connection header received
0809 15:18:53.232290 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:53.232297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:53.232355 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:53.232473 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:53.233405 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:53.233411 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:53.233416 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:53.233435 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:53.233484 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:53.233509 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:53.233646 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:53.233708 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 15:18:53.233789 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:18:53.241932 6351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ea66ad862e428aaacfd118d12be21f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44567 } }
I20260809 15:18:53.242357 6351 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:53.242436 6351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37ea66ad862e428aaacfd118d12be21f, State: Initialized, Role: FOLLOWER
I20260809 15:18:53.242992 6351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ea66ad862e428aaacfd118d12be21f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44567 } }
I20260809 15:18:53.243216 6351 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:53.243337 6351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:53.243465 6351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:53.246289 6351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ea66ad862e428aaacfd118d12be21f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44567 } }
I20260809 15:18:53.246758 6351 leader_election.cc:302] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37ea66ad862e428aaacfd118d12be21f; no voters:
I20260809 15:18:53.247329 6351 leader_election.cc:288] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:53.247480 6361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:53.248365 6361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [term 1 LEADER]: Becoming Leader. State: Replica: 37ea66ad862e428aaacfd118d12be21f, State: Running, Role: LEADER
I20260809 15:18:53.248998 6361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ea66ad862e428aaacfd118d12be21f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44567 } }
I20260809 15:18:53.249840 6351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:53.251911 6363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37ea66ad862e428aaacfd118d12be21f. Latest consensus state: current_term: 1 leader_uuid: "37ea66ad862e428aaacfd118d12be21f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ea66ad862e428aaacfd118d12be21f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44567 } } }
I20260809 15:18:53.252542 6363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:53.252550 6362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37ea66ad862e428aaacfd118d12be21f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37ea66ad862e428aaacfd118d12be21f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44567 } } }
I20260809 15:18:53.252967 6362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:53.254220 6366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:53.259546 6366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:53.269404 6366 catalog_manager.cc:1347] Generated new cluster ID: 8bd93d6c06e142a0987dca3ce623da04
I20260809 15:18:53.269512 6366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:53.291810 6366 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:53.292318 6366 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:53.303520 6366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37ea66ad862e428aaacfd118d12be21f: Generated new TSK 0
I20260809 15:18:53.304534 6366 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:53.314589 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6279
W20260809 15:18:53.315680 6345 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.214:46261
W20260809 15:18:53.316488 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6279: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:18:53.316990 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 6279
2026-08-09T15:18:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1987 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:53 dist-test-slave-8w9v krb5kdc[6382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:53 dist-test-slave-8w9v krb5kdc[6382](info): set up 2 sockets
Aug 09 15:18:53 dist-test-slave-8w9v krb5kdc[6382](info): commencing operation
krb5kdc: starting...
W20260809 15:18:54.485116 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:54 dist-test-slave-8w9v krb5kdc[6382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:54.781641 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:55.114663 6396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:55.114939 6396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:55.115106 6396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:55.122234 6396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:55.122337 6396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:55.122422 6396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:55.122529 6396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:55.122612 6396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:55.132333 6396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39553
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:55.133620 6396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:55.134925 6396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:55.146831 6401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:55.146781 6404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:55.147044 6406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:55.147799 6396 server_base.cc:1034] running on GCE node
I20260809 15:18:55.148603 6396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:55.149959 6396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:55.151170 6396 hybrid_clock.cc:648] HybridClock initialized: now 1786288735151133 us; error 28 us; skew 500 ppm
I20260809 15:18:55.151521 6396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:55 dist-test-slave-8w9v krb5kdc[6382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288735, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:55.154675 6396 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:55.156213 6396 webserver.cc:460] Webserver started at http://127.0.0.1:34969/ using document root <none> and password file <none>
I20260809 15:18:55.156880 6396 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:55.157028 6396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:55.157370 6396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:55.160346 6396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "60d4533b77d3458d91a7dd5c3e4d284e"
format_stamp: "Formatted at 2026-08-09 15:18:55 on dist-test-slave-8w9v"
server_key: "f8d4f4dfc842b29c1b21cf00cd42b10f"
server_key_iv: "1b9eb56e9f870fb09f94ed056004b78a"
server_key_version: "encryptionkey@0"
I20260809 15:18:55.161140 6396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "60d4533b77d3458d91a7dd5c3e4d284e"
format_stamp: "Formatted at 2026-08-09 15:18:55 on dist-test-slave-8w9v"
server_key: "f8d4f4dfc842b29c1b21cf00cd42b10f"
server_key_iv: "1b9eb56e9f870fb09f94ed056004b78a"
server_key_version: "encryptionkey@0"
I20260809 15:18:55.166080 6396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:18:55.169526 6414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:55.170766 6396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:18:55.170948 6396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "60d4533b77d3458d91a7dd5c3e4d284e"
format_stamp: "Formatted at 2026-08-09 15:18:55 on dist-test-slave-8w9v"
server_key: "f8d4f4dfc842b29c1b21cf00cd42b10f"
server_key_iv: "1b9eb56e9f870fb09f94ed056004b78a"
server_key_version: "encryptionkey@0"
I20260809 15:18:55.171144 6396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:55.188206 6396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:55.191999 6396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:55.192330 6396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:55.210853 6396 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39553
I20260809 15:18:55.210892 6475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39553 every 8 connection(s)
I20260809 15:18:55.212225 6396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:55.215778 6477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:55.220202 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 6396
I20260809 15:18:55.220383 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:55.220963 15406 external_mini_cluster.cc:1428] Setting key d2fedef5e26898b6310be52ae7689b25
I20260809 15:18:55.227330 6477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e: Bootstrap starting.
I20260809 15:18:55.231050 6477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:55.232265 6477 log.cc:824] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:55.234926 6477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e: No bootstrap required, opened a new log
Aug 09 15:18:55 dist-test-slave-8w9v krb5kdc[6382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:18:55.244800 6477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4533b77d3458d91a7dd5c3e4d284e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39553 } }
I20260809 15:18:55.245191 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:55.222735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:46795 (local address 127.0.0.1:39553)
0809 15:18:55.223517 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:55.223533 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:55.223576 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:18:55.225777 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:55.225800 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:55.226183 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:55.226491 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:55.228484 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:55.229799 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:55.231605 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:55.232348 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:55.232549 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:55.237202 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:55.237210 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:55.237226 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:55.237261 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:55.240517 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:55.241444 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:55.241451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:55.241457 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:55.241641 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:55.242153 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:55.242156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:55.242159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:55.242682 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:55.242951 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:55.243496 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 15:18:55.243867 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":355,"threads_started":1}
I20260809 15:18:55.245332 6477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:55.245456 6477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d4533b77d3458d91a7dd5c3e4d284e, State: Initialized, Role: FOLLOWER
I20260809 15:18:55.246057 6477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4533b77d3458d91a7dd5c3e4d284e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39553 } }
I20260809 15:18:55.246311 6477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:55.246474 6477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:55.246692 6477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:55.249624 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:55.249884 6477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4533b77d3458d91a7dd5c3e4d284e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39553 } }
I20260809 15:18:55.250428 6477 leader_election.cc:302] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60d4533b77d3458d91a7dd5c3e4d284e; no voters:
I20260809 15:18:55.251084 6477 leader_election.cc:288] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:55.251233 6482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:55.252509 6482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [term 1 LEADER]: Becoming Leader. State: Replica: 60d4533b77d3458d91a7dd5c3e4d284e, State: Running, Role: LEADER
I20260809 15:18:55.253122 6482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4533b77d3458d91a7dd5c3e4d284e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39553 } }
I20260809 15:18:55.253453 6477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:55.256167 6485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60d4533b77d3458d91a7dd5c3e4d284e. Latest consensus state: current_term: 1 leader_uuid: "60d4533b77d3458d91a7dd5c3e4d284e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4533b77d3458d91a7dd5c3e4d284e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39553 } } }
I20260809 15:18:55.256562 6485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:55.257117 6483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60d4533b77d3458d91a7dd5c3e4d284e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60d4533b77d3458d91a7dd5c3e4d284e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39553 } } }
I20260809 15:18:55.257411 6483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:55.259610 6494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:55.265239 6494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:55.279348 6480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:55.256179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:39293 (local address 127.0.0.1:39553)
0809 15:18:55.256370 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:55.256377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:55.260352 (+ 3975us) server_negotiation.cc:407] Connection header received
0809 15:18:55.260591 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:55.260599 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:55.260704 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:55.260813 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:55.262796 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:55.263981 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:55.265774 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:55.266448 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:55.266547 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:55.270066 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:55.270072 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:55.270076 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:18:55.270116 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:55.274595 (+ 4479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:55.275874 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:55.275882 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:55.275889 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:55.276091 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:18:55.276870 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:18:55.276878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:18:55.276887 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:18:55.277440 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:55.277754 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:55.278598 (+ 844us) server_negotiation.cc:299] Negotiation successful
0809 15:18:55.278829 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:18:55.283807 6494 catalog_manager.cc:1347] Generated new cluster ID: 41f7fc8e6b584b03bc737b740bc791f3
I20260809 15:18:55.283934 6494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:55.296954 6494 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:55.298673 6494 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:55.306739 6494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60d4533b77d3458d91a7dd5c3e4d284e: Generated new TSK 0
I20260809 15:18:55.307670 6494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:55.313827 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6396
I20260809 15:18:55.315069 6469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:18:55.402158 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 6396
W20260809 15:18:55.420859 6380 connection.cc:535] client connection to 127.0.0.1:39553 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39553): Connection reset by peer (error 104)
2026-08-09T15:18:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2082 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:55 dist-test-slave-8w9v krb5kdc[6514](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:55 dist-test-slave-8w9v krb5kdc[6514](info): set up 2 sockets
Aug 09 15:18:55 dist-test-slave-8w9v krb5kdc[6514](info): commencing operation
krb5kdc: starting...
W20260809 15:18:56.552853 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:56 dist-test-slave-8w9v krb5kdc[6514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:56.853668 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:57.184258 6529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:57.184525 6529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:57.184615 6529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:57.191542 6529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:57.191658 6529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:57.191704 6529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:57.191763 6529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:57.191811 6529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:57.201074 6529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:57.202291 6529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:57.203573 6529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:57.215649 6535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:57.215643 6536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:57.215654 6538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:57.216286 6529 server_base.cc:1034] running on GCE node
I20260809 15:18:57.217121 6529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:57.218462 6529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:57.219688 6529 hybrid_clock.cc:648] HybridClock initialized: now 1786288737219650 us; error 52 us; skew 500 ppm
I20260809 15:18:57.220031 6529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:57 dist-test-slave-8w9v krb5kdc[6514](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288737, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:57.223204 6529 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:57.224680 6529 webserver.cc:460] Webserver started at http://127.0.0.1:39675/ using document root <none> and password file <none>
I20260809 15:18:57.225346 6529 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:57.225512 6529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:57.225818 6529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:57.228834 6529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "1932d4bc6bc14777ae6c82766660ec90"
format_stamp: "Formatted at 2026-08-09 15:18:57 on dist-test-slave-8w9v"
server_key: "09327d22ecd10d7a055e931a91e8f4ba"
server_key_iv: "a9c0b6610112c2f4188e758a51ed8441"
server_key_version: "encryptionkey@0"
I20260809 15:18:57.229614 6529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "1932d4bc6bc14777ae6c82766660ec90"
format_stamp: "Formatted at 2026-08-09 15:18:57 on dist-test-slave-8w9v"
server_key: "09327d22ecd10d7a055e931a91e8f4ba"
server_key_iv: "a9c0b6610112c2f4188e758a51ed8441"
server_key_version: "encryptionkey@0"
I20260809 15:18:57.234683 6529 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:18:57.238111 6546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:57.239360 6529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:18:57.239548 6529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "1932d4bc6bc14777ae6c82766660ec90"
format_stamp: "Formatted at 2026-08-09 15:18:57 on dist-test-slave-8w9v"
server_key: "09327d22ecd10d7a055e931a91e8f4ba"
server_key_iv: "a9c0b6610112c2f4188e758a51ed8441"
server_key_version: "encryptionkey@0"
I20260809 15:18:57.239744 6529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:57.270741 6529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:57.275276 6529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:57.275586 6529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:57.294036 6529 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37245
I20260809 15:18:57.294081 6603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37245 every 8 connection(s)
I20260809 15:18:57.295449 6529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:57.299256 6604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:57.300364 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 6529
I20260809 15:18:57.300570 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:57.301023 15406 external_mini_cluster.cc:1428] Setting key 23185708c6fb27502f74b930bbc2de90
I20260809 15:18:57.313519 6604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90: Bootstrap starting.
I20260809 15:18:57.314867 6607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:57.303091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:54719 (local address 127.0.0.1:37245)
0809 15:18:57.303873 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:57.303884 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:57.303935 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:18:57.305723 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:57.305744 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:57.306141 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:57.306494 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:57.307084 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:57.308421 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:57.310354 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:57.311035 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:57.311197 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:57.312542 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:57.312549 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:57.312562 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:57.313075 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:57.313176 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:57.313212 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:57.313406 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:57.313732 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 15:18:57.313920 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":414,"threads_started":1}
I20260809 15:18:57.317101 6604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:57.318446 6604 log.cc:824] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:57.318969 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:57.321331 6604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90: No bootstrap required, opened a new log
I20260809 15:18:57.330603 6607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:57.323704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:45603 (local address 127.0.0.1:37245)
0809 15:18:57.323898 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:57.323903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:57.324453 (+ 550us) server_negotiation.cc:407] Connection header received
0809 15:18:57.324645 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:57.324648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:57.324712 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:57.324789 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:57.325368 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:57.326117 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:57.328007 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:18:57.328499 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:18:57.328659 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:18:57.329850 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:57.329855 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:57.329858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:57.329876 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:57.329951 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:57.329969 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:57.330095 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:57.330300 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 15:18:57.330351 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:18:57.331146 6604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932d4bc6bc14777ae6c82766660ec90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260809 15:18:57.331634 6604 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:57.331750 6604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1932d4bc6bc14777ae6c82766660ec90, State: Initialized, Role: FOLLOWER
I20260809 15:18:57.332434 6604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932d4bc6bc14777ae6c82766660ec90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260809 15:18:57.332683 6604 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:57.332840 6604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:57.333001 6604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:57.335660 6604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932d4bc6bc14777ae6c82766660ec90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260809 15:18:57.336261 6604 leader_election.cc:302] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1932d4bc6bc14777ae6c82766660ec90; no voters:
I20260809 15:18:57.336808 6604 leader_election.cc:288] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:57.336961 6615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:57.337751 6615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [term 1 LEADER]: Becoming Leader. State: Replica: 1932d4bc6bc14777ae6c82766660ec90, State: Running, Role: LEADER
I20260809 15:18:57.338340 6615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932d4bc6bc14777ae6c82766660ec90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } }
I20260809 15:18:57.339015 6604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:57.341014 6617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1932d4bc6bc14777ae6c82766660ec90. Latest consensus state: current_term: 1 leader_uuid: "1932d4bc6bc14777ae6c82766660ec90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932d4bc6bc14777ae6c82766660ec90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } } }
I20260809 15:18:57.341136 6616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1932d4bc6bc14777ae6c82766660ec90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1932d4bc6bc14777ae6c82766660ec90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37245 } } }
I20260809 15:18:57.341454 6617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:57.341663 6616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:57.343927 6621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:57.350598 6621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:57.359691 6621 catalog_manager.cc:1347] Generated new cluster ID: 338ebfd6c6dc419e8f0346225de863f8
I20260809 15:18:57.359819 6621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:57.368980 6621 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:57.370430 6621 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:57.382936 6621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1932d4bc6bc14777ae6c82766660ec90: Generated new TSK 0
I20260809 15:18:57.383680 6621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:57.398025 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6529
W20260809 15:18:57.399410 6599 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.214:54719
W20260809 15:18:57.400321 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6529: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:18:57.400475 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 6529
W20260809 15:18:57.419203 6512 connection.cc:535] client connection to 127.0.0.1:37245 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37245): Connection reset by peer (error 104)
2026-08-09T15:18:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1997 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:57 dist-test-slave-8w9v krb5kdc[6637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:57 dist-test-slave-8w9v krb5kdc[6637](info): set up 2 sockets
Aug 09 15:18:57 dist-test-slave-8w9v krb5kdc[6637](info): commencing operation
krb5kdc: starting...
W20260809 15:18:58.549474 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:18:58 dist-test-slave-8w9v krb5kdc[6637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:18:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:18:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:18:58.855404 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:18:59.193270 6651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:18:59.193545 6651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:18:59.193655 6651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:18:59.200783 6651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:18:59.200894 6651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:18:59.200939 6651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:18:59.201005 6651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:18:59.201049 6651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:18:59.210621 6651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:18:59.211840 6651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:18:59.213150 6651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:18:59.225696 6656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:59.225765 6657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:18:59.225690 6659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:18:59.226766 6651 server_base.cc:1034] running on GCE node
I20260809 15:18:59.227538 6651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:18:59.229014 6651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:18:59.230258 6651 hybrid_clock.cc:648] HybridClock initialized: now 1786288739230218 us; error 59 us; skew 500 ppm
I20260809 15:18:59.230602 6651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:18:59 dist-test-slave-8w9v krb5kdc[6637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288739, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:18:59.239089 6651 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:18:59.240753 6651 webserver.cc:460] Webserver started at http://127.0.0.1:34785/ using document root <none> and password file <none>
I20260809 15:18:59.241506 6651 fs_manager.cc:359] Metadata directory not provided
I20260809 15:18:59.241678 6651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:18:59.241999 6651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:18:59.245221 6651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "3f651b04373f49a8b5f631883bcdea83"
format_stamp: "Formatted at 2026-08-09 15:18:59 on dist-test-slave-8w9v"
server_key: "8b807324629744fab446c0042cbc586e"
server_key_iv: "9faec0f70fe6bc58fd0d2af37d3d86f9"
server_key_version: "encryptionkey@0"
I20260809 15:18:59.246042 6651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "3f651b04373f49a8b5f631883bcdea83"
format_stamp: "Formatted at 2026-08-09 15:18:59 on dist-test-slave-8w9v"
server_key: "8b807324629744fab446c0042cbc586e"
server_key_iv: "9faec0f70fe6bc58fd0d2af37d3d86f9"
server_key_version: "encryptionkey@0"
I20260809 15:18:59.251449 6651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:18:59.255544 6666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:18:59.256968 6651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 15:18:59.257169 6651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "3f651b04373f49a8b5f631883bcdea83"
format_stamp: "Formatted at 2026-08-09 15:18:59 on dist-test-slave-8w9v"
server_key: "8b807324629744fab446c0042cbc586e"
server_key_iv: "9faec0f70fe6bc58fd0d2af37d3d86f9"
server_key_version: "encryptionkey@0"
I20260809 15:18:59.257395 6651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:18:59.263345 6651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:18:59.267838 6651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:18:59.268204 6651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:18:59.287166 6651 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39163
I20260809 15:18:59.287216 6721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39163 every 8 connection(s)
I20260809 15:18:59.288686 6651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:18:59.292531 6722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:18:59.294121 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 6651
I20260809 15:18:59.294296 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:18:59.294811 15406 external_mini_cluster.cc:1428] Setting key a1aa590e48bd6ed09e6cea2e06967244
I20260809 15:18:59.303143 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:59.296792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:53213 (local address 127.0.0.1:39163)
0809 15:18:59.297605 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:59.297620 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:59.297671 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:18:59.299372 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:59.299404 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:59.299842 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:59.300330 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:59.301256 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:59.301263 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:59.301279 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:59.301295 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:59.301364 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:59.301388 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:59.301602 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:59.301734 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 15:18:59.301944 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":353,"threads_started":1}
I20260809 15:18:59.306806 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:18:59.306809 6722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83: Bootstrap starting.
I20260809 15:18:59.310616 6722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83: Neither blocks nor log segments found. Creating new log.
I20260809 15:18:59.311995 6722 log.cc:824] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83: Log is configured to *not* fsync() on all Append() calls
I20260809 15:18:59.314762 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:18:59.312088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.214:36343 (local address 127.0.0.1:39163)
0809 15:18:59.312269 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:18:59.312276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:18:59.312765 (+ 489us) server_negotiation.cc:407] Connection header received
0809 15:18:59.312962 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:18:59.312967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:18:59.313043 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:18:59.313162 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:18:59.314112 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:18:59.314118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:18:59.314123 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:18:59.314140 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:18:59.314185 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:18:59.314200 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:18:59.314330 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:18:59.314456 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 15:18:59.314512 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:18:59.314939 6722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83: No bootstrap required, opened a new log
I20260809 15:18:59.323048 6722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f651b04373f49a8b5f631883bcdea83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39163 } }
I20260809 15:18:59.323434 6722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:18:59.323558 6722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f651b04373f49a8b5f631883bcdea83, State: Initialized, Role: FOLLOWER
I20260809 15:18:59.324209 6722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f651b04373f49a8b5f631883bcdea83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39163 } }
I20260809 15:18:59.324479 6722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:18:59.324635 6722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:18:59.324813 6722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:18:59.327133 6722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f651b04373f49a8b5f631883bcdea83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39163 } }
I20260809 15:18:59.327636 6722 leader_election.cc:302] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f651b04373f49a8b5f631883bcdea83; no voters:
I20260809 15:18:59.328291 6722 leader_election.cc:288] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:18:59.328442 6733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:18:59.329115 6733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [term 1 LEADER]: Becoming Leader. State: Replica: 3f651b04373f49a8b5f631883bcdea83, State: Running, Role: LEADER
I20260809 15:18:59.329625 6733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f651b04373f49a8b5f631883bcdea83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39163 } }
I20260809 15:18:59.330348 6722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:18:59.332721 6735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f651b04373f49a8b5f631883bcdea83. Latest consensus state: current_term: 1 leader_uuid: "3f651b04373f49a8b5f631883bcdea83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f651b04373f49a8b5f631883bcdea83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39163 } } }
I20260809 15:18:59.333091 6735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:59.333276 6734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f651b04373f49a8b5f631883bcdea83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f651b04373f49a8b5f631883bcdea83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39163 } } }
I20260809 15:18:59.334206 6734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83 [sys.catalog]: This master's current role is: LEADER
I20260809 15:18:59.336220 6741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:18:59.344385 6741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:18:59.353286 6741 catalog_manager.cc:1347] Generated new cluster ID: 0f3bc0a3d9fd466b86052cd199ac82bc
I20260809 15:18:59.353420 6741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:18:59.379269 6741 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:18:59.379765 6741 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:18:59.393631 6741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f651b04373f49a8b5f631883bcdea83: Generated new TSK 0
I20260809 15:18:59.394713 6741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:18:59.402639 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6651
W20260809 15:18:59.404075 6716 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.214:53213
W20260809 15:18:59.404898 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6651: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:18:59.405102 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 6651
2026-08-09T15:18:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2008 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:18:59 dist-test-slave-8w9v krb5kdc[6757](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:18:59 dist-test-slave-8w9v krb5kdc[6757](info): set up 2 sockets
Aug 09 15:18:59 dist-test-slave-8w9v krb5kdc[6757](info): commencing operation
krb5kdc: starting...
W20260809 15:19:00.585182 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:19:00 dist-test-slave-8w9v krb5kdc[6757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:19:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:19:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:19:00.887604 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:44459
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:42219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:44459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:19:01.229085 6770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:01.229384 6770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:01.229475 6770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:01.236553 6770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:01.236653 6770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:01.236699 6770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:01.236760 6770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:01.236804 6770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:01.246443 6770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:42219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:44459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:44459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:01.247677 6770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:01.248980 6770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:01.260985 6775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:01.260947 6778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:01.261018 6776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:01.261696 6770 server_base.cc:1034] running on GCE node
I20260809 15:19:01.262475 6770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:01.263837 6770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:01.265070 6770 hybrid_clock.cc:648] HybridClock initialized: now 1786288741265031 us; error 54 us; skew 500 ppm
I20260809 15:19:01.265443 6770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:01 dist-test-slave-8w9v krb5kdc[6757](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288741, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:01.268733 6770 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:19:01.270179 6770 webserver.cc:460] Webserver started at http://127.15.11.190:38355/ using document root <none> and password file <none>
I20260809 15:19:01.270836 6770 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:01.270983 6770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:01.271283 6770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:01.274284 6770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "de9d5d24af4b44a88cc5831c4df27a39"
format_stamp: "Formatted at 2026-08-09 15:19:01 on dist-test-slave-8w9v"
server_key: "351fe4fa20b00e878d0dcc087df55877"
server_key_iv: "cb97013c579ccae51afd775eccf83425"
server_key_version: "encryptionkey@0"
I20260809 15:19:01.275111 6770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "de9d5d24af4b44a88cc5831c4df27a39"
format_stamp: "Formatted at 2026-08-09 15:19:01 on dist-test-slave-8w9v"
server_key: "351fe4fa20b00e878d0dcc087df55877"
server_key_iv: "cb97013c579ccae51afd775eccf83425"
server_key_version: "encryptionkey@0"
I20260809 15:19:01.280113 6770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:19:01.283349 6786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:01.284684 6770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:19:01.284874 6770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "de9d5d24af4b44a88cc5831c4df27a39"
format_stamp: "Formatted at 2026-08-09 15:19:01 on dist-test-slave-8w9v"
server_key: "351fe4fa20b00e878d0dcc087df55877"
server_key_iv: "cb97013c579ccae51afd775eccf83425"
server_key_version: "encryptionkey@0"
I20260809 15:19:01.285070 6770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:01.302999 6770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:01.307299 6770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:01.307597 6770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:01.326452 6770 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:44459
I20260809 15:19:01.326474 6838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:44459 every 8 connection(s)
I20260809 15:19:01.327800 6770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:19:01.331457 6839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:01.336102 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 6770
I20260809 15:19:01.336335 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:19:01.336987 15406 external_mini_cluster.cc:1428] Setting key 1f35ced00a9a24ada727e62257df725d
I20260809 15:19:01.342722 6839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39: Bootstrap starting.
I20260809 15:19:01.345916 6839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:01.346920 6839 log.cc:824] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:01.349792 6839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39: No bootstrap required, opened a new log
Aug 09 15:19:01 dist-test-slave-8w9v krb5kdc[6757](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:19:01.358328 6839 raft_consensus.cc:348] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9d5d24af4b44a88cc5831c4df27a39" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44459 } }
I20260809 15:19:01.358814 6839 raft_consensus.cc:374] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:19:01.358920 6839 raft_consensus.cc:729] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9d5d24af4b44a88cc5831c4df27a39, State: Initialized, Role: FOLLOWER
I20260809 15:19:01.359591 6839 consensus_queue.cc:260] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9d5d24af4b44a88cc5831c4df27a39" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44459 } }
I20260809 15:19:01.359840 6839 raft_consensus.cc:388] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:19:01.359867 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:01.338794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44553 (local address 127.15.11.190:44459)
0809 15:19:01.339630 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:01.339641 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:01.339684 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:19:01.341512 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:01.341533 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:01.341874 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:01.342246 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:01.343783 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:01.345250 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:01.346710 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:01.347543 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:01.347750 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:01.352039 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:01.352046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:01.352062 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:01.352098 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:01.355258 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:01.356172 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:01.356181 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:01.356191 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:01.356364 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:01.356904 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:01.356908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:01.356910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:01.357434 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:01.357788 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:01.358263 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 15:19:01.358631 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":388,"threads_started":1}
I20260809 15:19:01.360044 6839 raft_consensus.cc:482] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:19:01.360298 6839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:01.362807 6839 raft_consensus.cc:504] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9d5d24af4b44a88cc5831c4df27a39" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44459 } }
I20260809 15:19:01.363325 6839 leader_election.cc:302] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de9d5d24af4b44a88cc5831c4df27a39; no voters:
I20260809 15:19:01.363823 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:19:01.364115 6839 leader_election.cc:288] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:19:01.364249 6845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:19:01.365370 6845 raft_consensus.cc:686] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [term 1 LEADER]: Becoming Leader. State: Replica: de9d5d24af4b44a88cc5831c4df27a39, State: Running, Role: LEADER
I20260809 15:19:01.365988 6845 consensus_queue.cc:237] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9d5d24af4b44a88cc5831c4df27a39" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44459 } }
I20260809 15:19:01.366649 6839 sys_catalog.cc:564] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:01.369257 6846 sys_catalog.cc:455] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de9d5d24af4b44a88cc5831c4df27a39. Latest consensus state: current_term: 1 leader_uuid: "de9d5d24af4b44a88cc5831c4df27a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9d5d24af4b44a88cc5831c4df27a39" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44459 } } }
I20260809 15:19:01.369756 6846 sys_catalog.cc:458] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:01.370280 6847 sys_catalog.cc:455] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de9d5d24af4b44a88cc5831c4df27a39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de9d5d24af4b44a88cc5831c4df27a39" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 44459 } } }
I20260809 15:19:01.370620 6847 sys_catalog.cc:458] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:01.377478 6853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:19:01.385932 6853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:19:01.402634 6843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:01.378735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44219 (local address 127.15.11.190:44459)
0809 15:19:01.378981 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:01.378991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:01.379327 (+ 336us) server_negotiation.cc:407] Connection header received
0809 15:19:01.379639 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:01.379645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:01.379723 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:01.379849 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:01.382220 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:01.383887 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:01.386304 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:01.387284 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:01.387437 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:01.391902 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:01.391914 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:01.391922 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:01.391979 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:01.396962 (+ 4983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:01.398339 (+ 1377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:01.398347 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:01.398355 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:01.398538 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:01.399274 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:01.399282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:01.399288 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:01.399814 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:01.400185 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:01.401800 (+ 1615us) server_negotiation.cc:299] Negotiation successful
0809 15:19:01.402068 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 15:19:01.405084 6853 catalog_manager.cc:1347] Generated new cluster ID: 8f28fc84c6ec4f7f887f73a654b0f70c
I20260809 15:19:01.405515 6853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:19:01.435133 6853 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:19:01.437075 6853 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:19:01.447938 6853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de9d5d24af4b44a88cc5831c4df27a39: Generated new TSK 0
I20260809 15:19:01.448752 6853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:19:01.458654 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6770
I20260809 15:19:01.460397 6834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:19:01.552938 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 6770
W20260809 15:19:01.575088 6755 connection.cc:535] client connection to 127.15.11.190:44459 recv error: Network error: failed to read from TLS socket (remote: 127.15.11.190:44459): Connection reset by peer (error 104)
2026-08-09T15:19:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2147 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:19:01 dist-test-slave-8w9v krb5kdc[6874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:19:01 dist-test-slave-8w9v krb5kdc[6874](info): set up 2 sockets
Aug 09 15:19:01 dist-test-slave-8w9v krb5kdc[6874](info): commencing operation
krb5kdc: starting...
W20260809 15:19:02.729622 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:19:02 dist-test-slave-8w9v krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:19:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:19:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:19:03.024452 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:43475
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:34023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:43475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:19:03.373809 6890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:03.374182 6890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:03.374379 6890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:03.381687 6890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:03.381861 6890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:03.381987 6890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:03.382110 6890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:03.382206 6890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:03.391979 6890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:34023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:43475
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:43475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:03.393450 6890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:03.394898 6890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:03.407845 6897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:03.407956 6898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:03.408052 6903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:03.408768 6890 server_base.cc:1034] running on GCE node
I20260809 15:19:03.409439 6890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:03.410771 6890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:03.412004 6890 hybrid_clock.cc:648] HybridClock initialized: now 1786288743411973 us; error 49 us; skew 500 ppm
I20260809 15:19:03.412411 6890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:03 dist-test-slave-8w9v krb5kdc[6874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288743, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:03.415603 6890 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:19:03.417083 6890 webserver.cc:460] Webserver started at http://127.15.11.190:40503/ using document root <none> and password file <none>
I20260809 15:19:03.417714 6890 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:03.417810 6890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:03.418052 6890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:03.420915 6890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "aa8073a1f09344f6a2c877c49dff959a"
format_stamp: "Formatted at 2026-08-09 15:19:03 on dist-test-slave-8w9v"
server_key: "3c817aa9364bb9cb230fb5025058e83d"
server_key_iv: "f3c5142b7d2a59b6c8aa91de9c1804dc"
server_key_version: "encryptionkey@0"
I20260809 15:19:03.421634 6890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "aa8073a1f09344f6a2c877c49dff959a"
format_stamp: "Formatted at 2026-08-09 15:19:03 on dist-test-slave-8w9v"
server_key: "3c817aa9364bb9cb230fb5025058e83d"
server_key_iv: "f3c5142b7d2a59b6c8aa91de9c1804dc"
server_key_version: "encryptionkey@0"
I20260809 15:19:03.426673 6890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:19:03.430094 6910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:03.431416 6890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:19:03.431568 6890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "aa8073a1f09344f6a2c877c49dff959a"
format_stamp: "Formatted at 2026-08-09 15:19:03 on dist-test-slave-8w9v"
server_key: "3c817aa9364bb9cb230fb5025058e83d"
server_key_iv: "f3c5142b7d2a59b6c8aa91de9c1804dc"
server_key_version: "encryptionkey@0"
I20260809 15:19:03.431722 6890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:03.462848 6890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:03.471668 6890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:03.471908 6890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:03.491010 6890 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:43475
I20260809 15:19:03.491034 6964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:43475 every 8 connection(s)
I20260809 15:19:03.492435 6890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:19:03.496045 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 6890
I20260809 15:19:03.496250 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:19:03.496820 6965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:03.496878 15406 external_mini_cluster.cc:1428] Setting key 16ab50831c6193e109259f287a72c217
I20260809 15:19:03.512108 6968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:03.498677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59683 (local address 127.15.11.190:43475)
0809 15:19:03.499484 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:03.499499 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:03.499544 (+ 45us) server_negotiation.cc:407] Connection header received
0809 15:19:03.501896 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:03.501925 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:03.502494 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:03.502942 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:03.503449 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:03.505489 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:03.506873 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:03.507953 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:03.508167 (+ 214us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:03.509431 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:03.509441 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:03.509459 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:19:03.510167 (+ 708us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:03.510270 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:19:03.510301 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:03.510465 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:03.510699 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 15:19:03.510860 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":312,"threads_started":1}
I20260809 15:19:03.512437 6965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a: Bootstrap starting.
I20260809 15:19:03.515637 6965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:03.515822 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:19:03.516801 6965 log.cc:824] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:03.519667 6965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a: No bootstrap required, opened a new log
I20260809 15:19:03.527532 6968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:03.520800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60751 (local address 127.15.11.190:43475)
0809 15:19:03.520974 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:03.520980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:03.521395 (+ 415us) server_negotiation.cc:407] Connection header received
0809 15:19:03.521532 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:03.521538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:03.521620 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:03.521729 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:03.522204 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:03.523389 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:03.524808 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:03.525527 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:03.525669 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:03.526750 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:03.526760 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:03.526770 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:19:03.526790 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:03.526843 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:19:03.526864 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:03.526972 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:03.527125 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 15:19:03.527173 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:19:03.529779 6965 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8073a1f09344f6a2c877c49dff959a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43475 } }
I20260809 15:19:03.530169 6965 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:19:03.530285 6965 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8073a1f09344f6a2c877c49dff959a, State: Initialized, Role: FOLLOWER
I20260809 15:19:03.530853 6965 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8073a1f09344f6a2c877c49dff959a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43475 } }
I20260809 15:19:03.531081 6965 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:19:03.531191 6965 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:19:03.531309 6965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:03.533838 6965 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8073a1f09344f6a2c877c49dff959a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43475 } }
I20260809 15:19:03.534272 6965 leader_election.cc:302] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa8073a1f09344f6a2c877c49dff959a; no voters:
I20260809 15:19:03.534926 6965 leader_election.cc:288] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:19:03.535068 6976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:19:03.537197 6965 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:03.540382 6976 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [term 1 LEADER]: Becoming Leader. State: Replica: aa8073a1f09344f6a2c877c49dff959a, State: Running, Role: LEADER
I20260809 15:19:03.540903 6976 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8073a1f09344f6a2c877c49dff959a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43475 } }
I20260809 15:19:03.543534 6977 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa8073a1f09344f6a2c877c49dff959a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8073a1f09344f6a2c877c49dff959a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43475 } } }
I20260809 15:19:03.543869 6977 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:03.544773 6979 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa8073a1f09344f6a2c877c49dff959a. Latest consensus state: current_term: 1 leader_uuid: "aa8073a1f09344f6a2c877c49dff959a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8073a1f09344f6a2c877c49dff959a" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 43475 } } }
I20260809 15:19:03.545228 6979 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:03.547979 6989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:19:03.552299 6989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:19:03.560740 6989 catalog_manager.cc:1347] Generated new cluster ID: af816987ce304145a662e03d34903696
I20260809 15:19:03.560851 6989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:19:03.581506 6989 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:19:03.583381 6989 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:19:03.603320 6989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa8073a1f09344f6a2c877c49dff959a: Generated new TSK 0
I20260809 15:19:03.604599 6989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:19:03.621836 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6890
W20260809 15:19:03.622951 6960 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59683
W20260809 15:19:03.623728 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 6890: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:19:03.623884 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 6890
W20260809 15:19:03.650399 6872 connection.cc:535] client connection to 127.15.11.190:43475 recv error: Network error: failed to read from TLS socket (remote: 127.15.11.190:43475): Connection reset by peer (error 104)
2026-08-09T15:19:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2076 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:19:03 dist-test-slave-8w9v krb5kdc[6999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:19:03 dist-test-slave-8w9v krb5kdc[6999](info): set up 2 sockets
Aug 09 15:19:03 dist-test-slave-8w9v krb5kdc[6999](info): commencing operation
krb5kdc: starting...
W20260809 15:19:04.796422 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:19:04 dist-test-slave-8w9v krb5kdc[6999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288744, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:19:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:19:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:19:05.089911 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:33239
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:33239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:19:05.427100 7014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:05.427400 7014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:05.427493 7014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:05.434540 7014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:05.434638 7014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:05.434689 7014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:05.434748 7014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:05.434791 7014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:05.443988 7014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:33239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:33239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:05.445228 7014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:05.446511 7014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:05.458438 7020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:05.458459 7022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:05.458459 7019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:05.459726 7014 server_base.cc:1034] running on GCE node
I20260809 15:19:05.460621 7014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:05.462543 7014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:05.463768 7014 hybrid_clock.cc:648] HybridClock initialized: now 1786288745463711 us; error 56 us; skew 500 ppm
I20260809 15:19:05.464190 7014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:05 dist-test-slave-8w9v krb5kdc[6999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288745, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:05.472437 7014 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:19:05.473901 7014 webserver.cc:460] Webserver started at http://127.15.11.190:39253/ using document root <none> and password file <none>
I20260809 15:19:05.474586 7014 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:05.474761 7014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:05.475082 7014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:05.478495 7014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "a35e0effa9a54ba384e900512b1a4262"
format_stamp: "Formatted at 2026-08-09 15:19:05 on dist-test-slave-8w9v"
server_key: "33f3d0fd4c0c19cf05560961e19720b5"
server_key_iv: "c5f5de5c3b52e823ea0434c605974cde"
server_key_version: "encryptionkey@0"
I20260809 15:19:05.479339 7014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "a35e0effa9a54ba384e900512b1a4262"
format_stamp: "Formatted at 2026-08-09 15:19:05 on dist-test-slave-8w9v"
server_key: "33f3d0fd4c0c19cf05560961e19720b5"
server_key_iv: "c5f5de5c3b52e823ea0434c605974cde"
server_key_version: "encryptionkey@0"
I20260809 15:19:05.484615 7014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260809 15:19:05.487926 7029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:05.489270 7014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:19:05.489485 7014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "a35e0effa9a54ba384e900512b1a4262"
format_stamp: "Formatted at 2026-08-09 15:19:05 on dist-test-slave-8w9v"
server_key: "33f3d0fd4c0c19cf05560961e19720b5"
server_key_iv: "c5f5de5c3b52e823ea0434c605974cde"
server_key_version: "encryptionkey@0"
I20260809 15:19:05.489696 7014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:05.495436 7014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:05.499749 7014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:05.500051 7014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:05.518880 7014 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:33239
I20260809 15:19:05.518916 7086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:33239 every 8 connection(s)
I20260809 15:19:05.520339 7014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:19:05.524072 7087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:05.527657 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 7014
I20260809 15:19:05.527850 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:19:05.528455 15406 external_mini_cluster.cc:1428] Setting key 19d9fad7662633e52f7c234bcbbd0a9f
I20260809 15:19:05.535966 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:05.530261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50977 (local address 127.15.11.190:33239)
0809 15:19:05.530985 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:05.530996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:05.531034 (+ 38us) server_negotiation.cc:407] Connection header received
0809 15:19:05.532698 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:05.532719 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:05.533037 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:05.533414 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:05.534464 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:05.534472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:05.534484 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:19:05.534497 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:05.534550 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:19:05.534566 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:05.534806 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:05.534965 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 15:19:05.535133 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":306,"threads_started":1}
I20260809 15:19:05.536598 7087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262: Bootstrap starting.
I20260809 15:19:05.539783 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:19:05.540104 7087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:05.541144 7087 log.cc:824] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:05.543913 7087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262: No bootstrap required, opened a new log
I20260809 15:19:05.546921 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:05.544639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58795 (local address 127.15.11.190:33239)
0809 15:19:05.544804 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:05.544814 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:05.545186 (+ 372us) server_negotiation.cc:407] Connection header received
0809 15:19:05.545292 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:05.545296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:05.545349 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:05.545468 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:05.546321 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:05.546326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:05.546332 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:19:05.546348 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:05.546408 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:19:05.546426 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:05.546530 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:05.546601 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 15:19:05.546648 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:19:05.552416 7087 raft_consensus.cc:348] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e0effa9a54ba384e900512b1a4262" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33239 } }
I20260809 15:19:05.552763 7087 raft_consensus.cc:374] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:19:05.552836 7087 raft_consensus.cc:729] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35e0effa9a54ba384e900512b1a4262, State: Initialized, Role: FOLLOWER
I20260809 15:19:05.553310 7087 consensus_queue.cc:260] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e0effa9a54ba384e900512b1a4262" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33239 } }
I20260809 15:19:05.553512 7087 raft_consensus.cc:388] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:19:05.553619 7087 raft_consensus.cc:482] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:19:05.553777 7087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:05.556367 7087 raft_consensus.cc:504] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e0effa9a54ba384e900512b1a4262" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33239 } }
I20260809 15:19:05.556815 7087 leader_election.cc:302] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a35e0effa9a54ba384e900512b1a4262; no voters:
I20260809 15:19:05.557500 7087 leader_election.cc:288] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:19:05.557647 7098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:19:05.558984 7098 raft_consensus.cc:686] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [term 1 LEADER]: Becoming Leader. State: Replica: a35e0effa9a54ba384e900512b1a4262, State: Running, Role: LEADER
I20260809 15:19:05.559557 7098 consensus_queue.cc:237] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e0effa9a54ba384e900512b1a4262" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33239 } }
I20260809 15:19:05.559883 7087 sys_catalog.cc:564] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:05.562425 7100 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a35e0effa9a54ba384e900512b1a4262. Latest consensus state: current_term: 1 leader_uuid: "a35e0effa9a54ba384e900512b1a4262" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e0effa9a54ba384e900512b1a4262" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33239 } } }
I20260809 15:19:05.562800 7100 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:05.562963 7099 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a35e0effa9a54ba384e900512b1a4262" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35e0effa9a54ba384e900512b1a4262" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 33239 } } }
I20260809 15:19:05.563273 7099 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:05.565461 7107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:19:05.569972 7107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:19:05.578739 7107 catalog_manager.cc:1347] Generated new cluster ID: cd48e24bb7404e909eaa76ca9bcd25f5
I20260809 15:19:05.578874 7107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:19:05.586697 7107 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:19:05.587121 7107 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:19:05.592782 7107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a35e0effa9a54ba384e900512b1a4262: Generated new TSK 0
I20260809 15:19:05.593576 7107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:19:05.608333 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7014
W20260809 15:19:05.609992 7082 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50977
W20260809 15:19:05.611513 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7014: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:19:05.611732 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 7014
2026-08-09T15:19:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1982 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:19:05 dist-test-slave-8w9v krb5kdc[7120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:19:05 dist-test-slave-8w9v krb5kdc[7120](info): set up 2 sockets
Aug 09 15:19:05 dist-test-slave-8w9v krb5kdc[7120](info): commencing operation
krb5kdc: starting...
W20260809 15:19:06.781999 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.006s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:19:06 dist-test-slave-8w9v krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:19:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:19:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:19:07.084596 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:45739
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:35003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:45739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:19:07.444443 7133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:07.444739 7133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:07.444854 7133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:07.452072 7133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:07.452291 7133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:07.452412 7133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:07.452535 7133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:07.452623 7133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:07.462751 7133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:35003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:45739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:45739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:07.464097 7133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:07.465559 7133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:07.478283 7138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:07.478219 7139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:07.478291 7142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:07.479067 7133 server_base.cc:1034] running on GCE node
I20260809 15:19:07.479911 7133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:07.481432 7133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:07.482683 7133 hybrid_clock.cc:648] HybridClock initialized: now 1786288747482646 us; error 52 us; skew 500 ppm
I20260809 15:19:07.483049 7133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:07 dist-test-slave-8w9v krb5kdc[7120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288747, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:07.486259 7133 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:19:07.487767 7133 webserver.cc:460] Webserver started at http://127.15.11.190:42219/ using document root <none> and password file <none>
I20260809 15:19:07.488551 7133 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:07.488714 7133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:07.489039 7133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:07.492034 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "6dfb33318ad648d685ffd8fe1dbcf484"
format_stamp: "Formatted at 2026-08-09 15:19:07 on dist-test-slave-8w9v"
server_key: "b36716671bfc180a18f07eb74f43832f"
server_key_iv: "f3b4f29e2c5d40a64627014e7ec0c1c8"
server_key_version: "encryptionkey@0"
I20260809 15:19:07.492892 7133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "6dfb33318ad648d685ffd8fe1dbcf484"
format_stamp: "Formatted at 2026-08-09 15:19:07 on dist-test-slave-8w9v"
server_key: "b36716671bfc180a18f07eb74f43832f"
server_key_iv: "f3b4f29e2c5d40a64627014e7ec0c1c8"
server_key_version: "encryptionkey@0"
I20260809 15:19:07.498059 7133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 15:19:07.501847 7150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:07.503064 7133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:19:07.503263 7133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "6dfb33318ad648d685ffd8fe1dbcf484"
format_stamp: "Formatted at 2026-08-09 15:19:07 on dist-test-slave-8w9v"
server_key: "b36716671bfc180a18f07eb74f43832f"
server_key_iv: "f3b4f29e2c5d40a64627014e7ec0c1c8"
server_key_version: "encryptionkey@0"
I20260809 15:19:07.503465 7133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:07.527397 7133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:07.531441 7133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:07.531759 7133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:07.550832 7133 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:45739
I20260809 15:19:07.550859 7206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:45739 every 8 connection(s)
I20260809 15:19:07.552279 7133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:19:07.554363 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 7133
I20260809 15:19:07.554548 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:19:07.555110 15406 external_mini_cluster.cc:1428] Setting key 994d3c4d31d6322032da549d6569a905
I20260809 15:19:07.556962 7207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 15:19:07 dist-test-slave-8w9v krb5kdc[7120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:19:07.571163 7207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484: Bootstrap starting.
I20260809 15:19:07.574631 7207 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:07.575697 7207 log.cc:824] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:07.578778 7207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484: No bootstrap required, opened a new log
I20260809 15:19:07.578986 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:07.556971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58807 (local address 127.15.11.190:45739)
0809 15:19:07.557837 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:07.557849 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:07.557893 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:19:07.559496 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:07.559517 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:07.559895 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:07.560371 (+ 476us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:07.562153 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:07.563497 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:07.565368 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:07.566027 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:07.566168 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:07.571394 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:07.571403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:07.571417 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:07.571452 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:07.574641 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:07.575373 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:07.575380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:07.575386 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:07.575569 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:07.576093 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:07.576102 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:07.576104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:07.576730 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:07.577077 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:07.577555 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 15:19:07.577974 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":377,"threads_started":1}
I20260809 15:19:07.582841 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:19:07.587193 7207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfb33318ad648d685ffd8fe1dbcf484" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45739 } }
I20260809 15:19:07.587682 7207 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:19:07.587812 7207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dfb33318ad648d685ffd8fe1dbcf484, State: Initialized, Role: FOLLOWER
I20260809 15:19:07.588532 7207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfb33318ad648d685ffd8fe1dbcf484" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45739 } }
I20260809 15:19:07.588840 7207 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:19:07.588997 7207 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:19:07.589182 7207 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:07.592080 7207 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfb33318ad648d685ffd8fe1dbcf484" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45739 } }
I20260809 15:19:07.592712 7207 leader_election.cc:302] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dfb33318ad648d685ffd8fe1dbcf484; no voters:
I20260809 15:19:07.593438 7207 leader_election.cc:288] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:19:07.593554 7217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:19:07.594853 7217 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [term 1 LEADER]: Becoming Leader. State: Replica: 6dfb33318ad648d685ffd8fe1dbcf484, State: Running, Role: LEADER
I20260809 15:19:07.595534 7217 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfb33318ad648d685ffd8fe1dbcf484" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45739 } }
I20260809 15:19:07.595855 7207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:07.602308 7218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dfb33318ad648d685ffd8fe1dbcf484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfb33318ad648d685ffd8fe1dbcf484" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45739 } } }
I20260809 15:19:07.602725 7218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:07.603252 7219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dfb33318ad648d685ffd8fe1dbcf484. Latest consensus state: current_term: 1 leader_uuid: "6dfb33318ad648d685ffd8fe1dbcf484" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dfb33318ad648d685ffd8fe1dbcf484" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 45739 } } }
I20260809 15:19:07.603605 7219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:07.607831 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:07.587293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48867 (local address 127.15.11.190:45739)
0809 15:19:07.587485 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:07.587497 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:07.588202 (+ 705us) server_negotiation.cc:407] Connection header received
0809 15:19:07.588449 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:07.588453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:07.588512 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:07.588595 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:07.590642 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:07.591453 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:07.593340 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:07.594223 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:07.594382 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:07.598154 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:07.598163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:07.598170 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:07.598221 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:07.603165 (+ 4944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:07.604344 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:07.604351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:07.604357 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:07.604586 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:07.605143 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:07.605150 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:07.605157 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:07.605621 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:07.607047 (+ 1426us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:07.607201 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 15:19:07.607428 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:19:07.616485 7228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:19:07.622066 7228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:19:07.630730 7228 catalog_manager.cc:1347] Generated new cluster ID: 537ac18370c24811bbac21683875666f
I20260809 15:19:07.630857 7228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:19:07.639734 7228 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:19:07.641571 7228 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:19:07.654529 7228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dfb33318ad648d685ffd8fe1dbcf484: Generated new TSK 0
I20260809 15:19:07.655318 7228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:19:07.664973 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7133
I20260809 15:19:07.666779 7201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:19:07.760667 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 7133
2026-08-09T15:19:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2147 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:19:07 dist-test-slave-8w9v krb5kdc[7244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:19:07 dist-test-slave-8w9v krb5kdc[7244](info): set up 2 sockets
Aug 09 15:19:07 dist-test-slave-8w9v krb5kdc[7244](info): commencing operation
krb5kdc: starting...
W20260809 15:19:08.939424 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:19:09 dist-test-slave-8w9v krb5kdc[7244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:19:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:19:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:19:09.233742 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:32907
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:38997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:32907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:19:09.573724 7259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:09.573998 7259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:09.574193 7259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:09.581172 7259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:09.581266 7259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:09.581378 7259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:09.581493 7259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:09.581584 7259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:09.591022 7259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:38997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:32907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:32907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:09.592327 7259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:09.593605 7259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:09.605633 7264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:09.605687 7267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:09.605594 7265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:09.606328 7259 server_base.cc:1034] running on GCE node
I20260809 15:19:09.607028 7259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:09.608359 7259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:09.609606 7259 hybrid_clock.cc:648] HybridClock initialized: now 1786288749609572 us; error 44 us; skew 500 ppm
I20260809 15:19:09.609956 7259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:09 dist-test-slave-8w9v krb5kdc[7244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288749, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:09.613037 7259 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:19:09.614493 7259 webserver.cc:460] Webserver started at http://127.15.11.190:41021/ using document root <none> and password file <none>
I20260809 15:19:09.615115 7259 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:09.615217 7259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:09.615463 7259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:09.618407 7259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "480a271fbc85453ca2904fe621a4fd5c"
format_stamp: "Formatted at 2026-08-09 15:19:09 on dist-test-slave-8w9v"
server_key: "a8913577235a889173fffc3a8a8d86b0"
server_key_iv: "399c6aa6b6834a9edc82f8cfd3ba8a58"
server_key_version: "encryptionkey@0"
I20260809 15:19:09.619117 7259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "480a271fbc85453ca2904fe621a4fd5c"
format_stamp: "Formatted at 2026-08-09 15:19:09 on dist-test-slave-8w9v"
server_key: "a8913577235a889173fffc3a8a8d86b0"
server_key_iv: "399c6aa6b6834a9edc82f8cfd3ba8a58"
server_key_version: "encryptionkey@0"
I20260809 15:19:09.623987 7259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 15:19:09.628463 7275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:09.629720 7259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:19:09.629920 7259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "480a271fbc85453ca2904fe621a4fd5c"
format_stamp: "Formatted at 2026-08-09 15:19:09 on dist-test-slave-8w9v"
server_key: "a8913577235a889173fffc3a8a8d86b0"
server_key_iv: "399c6aa6b6834a9edc82f8cfd3ba8a58"
server_key_version: "encryptionkey@0"
I20260809 15:19:09.630119 7259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:09.663254 7259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:09.667303 7259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:09.667618 7259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:09.686636 7259 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:32907
I20260809 15:19:09.686657 7332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:32907 every 8 connection(s)
I20260809 15:19:09.688105 7259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:19:09.691661 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 7259
I20260809 15:19:09.691854 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:19:09.692198 7333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:09.692457 15406 external_mini_cluster.cc:1428] Setting key 82bb1f5d0970a2bb59d5d610a0a7ac9a
I20260809 15:19:09.707644 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:09.694416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44015 (local address 127.15.11.190:32907)
0809 15:19:09.695299 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:09.695320 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:09.695371 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:19:09.697577 (+ 2206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:09.697604 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:09.698137 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:09.698565 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:09.699103 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:09.701209 (+ 2106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:09.702500 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:09.703535 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:09.703719 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:09.704925 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:09.704935 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:09.704952 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:19:09.705669 (+ 717us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:09.705771 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:19:09.705800 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:09.705908 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:09.706106 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 15:19:09.706350 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":359,"threads_started":1}
I20260809 15:19:09.707898 7333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c: Bootstrap starting.
I20260809 15:19:09.711786 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:19:09.711790 7333 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:09.713033 7333 log.cc:824] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:09.716451 7333 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c: No bootstrap required, opened a new log
I20260809 15:19:09.724282 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:09.717138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40795 (local address 127.15.11.190:32907)
0809 15:19:09.717345 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:09.717354 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:09.717878 (+ 524us) server_negotiation.cc:407] Connection header received
0809 15:19:09.718066 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:09.718070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:09.718127 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:09.718200 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:09.718822 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:09.719568 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:09.721518 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:09.722011 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:09.722105 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:09.723327 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:09.723334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:09.723338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:19:09.723356 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:09.723397 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:19:09.723427 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:09.723513 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:09.723698 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:19:09.723781 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:19:09.726507 7333 raft_consensus.cc:348] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480a271fbc85453ca2904fe621a4fd5c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 32907 } }
I20260809 15:19:09.726858 7333 raft_consensus.cc:374] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:19:09.726958 7333 raft_consensus.cc:729] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 480a271fbc85453ca2904fe621a4fd5c, State: Initialized, Role: FOLLOWER
I20260809 15:19:09.727643 7333 consensus_queue.cc:260] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480a271fbc85453ca2904fe621a4fd5c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 32907 } }
I20260809 15:19:09.727962 7333 raft_consensus.cc:388] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:19:09.728113 7333 raft_consensus.cc:482] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:19:09.728340 7333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:09.730401 7333 raft_consensus.cc:504] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480a271fbc85453ca2904fe621a4fd5c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 32907 } }
I20260809 15:19:09.730816 7333 leader_election.cc:302] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 480a271fbc85453ca2904fe621a4fd5c; no voters:
I20260809 15:19:09.731344 7333 leader_election.cc:288] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:19:09.731489 7344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:19:09.732218 7344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [term 1 LEADER]: Becoming Leader. State: Replica: 480a271fbc85453ca2904fe621a4fd5c, State: Running, Role: LEADER
I20260809 15:19:09.732718 7344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480a271fbc85453ca2904fe621a4fd5c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 32907 } }
I20260809 15:19:09.733552 7333 sys_catalog.cc:564] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:09.735458 7347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 480a271fbc85453ca2904fe621a4fd5c. Latest consensus state: current_term: 1 leader_uuid: "480a271fbc85453ca2904fe621a4fd5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480a271fbc85453ca2904fe621a4fd5c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 32907 } } }
I20260809 15:19:09.735849 7347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:09.736455 7345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "480a271fbc85453ca2904fe621a4fd5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480a271fbc85453ca2904fe621a4fd5c" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 32907 } } }
I20260809 15:19:09.736804 7345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:09.739859 7350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:19:09.745234 7350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:19:09.756623 7350 catalog_manager.cc:1347] Generated new cluster ID: 805105a1c2c14240bbbf522d86be345b
I20260809 15:19:09.756738 7350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:19:09.784775 7350 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:19:09.786155 7350 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:19:09.794657 7350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 480a271fbc85453ca2904fe621a4fd5c: Generated new TSK 0
I20260809 15:19:09.795428 7350 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:19:09.804831 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7259
W20260809 15:19:09.805992 7326 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44015
W20260809 15:19:09.806725 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7259: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:19:09.806880 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 7259
2026-08-09T15:19:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2045 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:19:09 dist-test-slave-8w9v krb5kdc[7367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:19:09 dist-test-slave-8w9v krb5kdc[7367](info): set up 2 sockets
Aug 09 15:19:09 dist-test-slave-8w9v krb5kdc[7367](info): commencing operation
krb5kdc: starting...
W20260809 15:19:10.989665 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.006s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:19:11 dist-test-slave-8w9v krb5kdc[7367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:19:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:19:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:19:11.290673 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:34713
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:34713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:19:11.627489 7383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:11.627763 7383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:11.627856 7383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:11.636363 7383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:11.636469 7383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:11.636525 7383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:11.636585 7383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:11.636629 7383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:11.646273 7383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:34713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:34713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:11.647575 7383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:11.648953 7383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:11.661149 7388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:11.661283 7389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:11.661249 7391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:11.662254 7383 server_base.cc:1034] running on GCE node
I20260809 15:19:11.663060 7383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:11.664470 7383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:11.665679 7383 hybrid_clock.cc:648] HybridClock initialized: now 1786288751665638 us; error 29 us; skew 500 ppm
I20260809 15:19:11.666038 7383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:11 dist-test-slave-8w9v krb5kdc[7367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288751, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:11.676803 7383 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:19:11.678290 7383 webserver.cc:460] Webserver started at http://127.15.11.190:45131/ using document root <none> and password file <none>
I20260809 15:19:11.679018 7383 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:11.679175 7383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:11.679510 7383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:11.682652 7383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "75e95fcb15334d7c925825c8db7e8202"
format_stamp: "Formatted at 2026-08-09 15:19:11 on dist-test-slave-8w9v"
server_key: "8a6e6af48debb8f67d77dc69a5ff1fe1"
server_key_iv: "c1767713a483ca2e7c030aa23ae3a8ab"
server_key_version: "encryptionkey@0"
I20260809 15:19:11.683454 7383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "75e95fcb15334d7c925825c8db7e8202"
format_stamp: "Formatted at 2026-08-09 15:19:11 on dist-test-slave-8w9v"
server_key: "8a6e6af48debb8f67d77dc69a5ff1fe1"
server_key_iv: "c1767713a483ca2e7c030aa23ae3a8ab"
server_key_version: "encryptionkey@0"
I20260809 15:19:11.688585 7383 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 15:19:11.692027 7398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:11.693315 7383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:19:11.693522 7383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "75e95fcb15334d7c925825c8db7e8202"
format_stamp: "Formatted at 2026-08-09 15:19:11 on dist-test-slave-8w9v"
server_key: "8a6e6af48debb8f67d77dc69a5ff1fe1"
server_key_iv: "c1767713a483ca2e7c030aa23ae3a8ab"
server_key_version: "encryptionkey@0"
I20260809 15:19:11.693727 7383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:11.699333 7383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:11.703152 7383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:11.703466 7383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:11.722160 7383 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:34713
I20260809 15:19:11.722195 7454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:34713 every 8 connection(s)
I20260809 15:19:11.723616 7383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:19:11.727269 7455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:11.727887 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 7383
I20260809 15:19:11.728073 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:19:11.728689 15406 external_mini_cluster.cc:1428] Setting key a04440dea7c192dc575df6438fd535cb
I20260809 15:19:11.736967 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:11.730488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37753 (local address 127.15.11.190:34713)
0809 15:19:11.731281 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:11.731292 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:11.731333 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:19:11.733559 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:11.733580 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:11.733921 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:11.734243 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:11.735373 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:11.735379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:11.735392 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:19:11.735406 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:11.735467 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:19:11.735492 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:11.735656 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:11.735796 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 15:19:11.736012 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":371,"threads_started":1}
I20260809 15:19:11.740455 7455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202: Bootstrap starting.
I20260809 15:19:11.740711 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:19:11.743510 7455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:11.744863 7455 log.cc:824] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:11.749111 7455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202: No bootstrap required, opened a new log
I20260809 15:19:11.749188 7458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:11.745355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46969 (local address 127.15.11.190:34713)
0809 15:19:11.745627 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:11.745636 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:11.746622 (+ 986us) server_negotiation.cc:407] Connection header received
0809 15:19:11.746826 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:11.746834 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:11.746913 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:11.747021 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:11.748192 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:11.748200 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:11.748226 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:19:11.748247 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:11.748297 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:19:11.748321 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:11.748446 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:11.748687 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 15:19:11.748779 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:19:11.758018 7455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e95fcb15334d7c925825c8db7e8202" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34713 } }
I20260809 15:19:11.758394 7455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:19:11.758518 7455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75e95fcb15334d7c925825c8db7e8202, State: Initialized, Role: FOLLOWER
I20260809 15:19:11.759109 7455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e95fcb15334d7c925825c8db7e8202" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34713 } }
I20260809 15:19:11.759353 7455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:19:11.759522 7455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:19:11.759699 7455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:11.761909 7455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e95fcb15334d7c925825c8db7e8202" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34713 } }
I20260809 15:19:11.762413 7455 leader_election.cc:302] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75e95fcb15334d7c925825c8db7e8202; no voters:
I20260809 15:19:11.763003 7455 leader_election.cc:288] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:19:11.763263 7465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:19:11.764264 7465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [term 1 LEADER]: Becoming Leader. State: Replica: 75e95fcb15334d7c925825c8db7e8202, State: Running, Role: LEADER
I20260809 15:19:11.764796 7465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e95fcb15334d7c925825c8db7e8202" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34713 } }
I20260809 15:19:11.765120 7455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:11.769605 7466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75e95fcb15334d7c925825c8db7e8202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e95fcb15334d7c925825c8db7e8202" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34713 } } }
I20260809 15:19:11.770083 7466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:11.770605 7467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75e95fcb15334d7c925825c8db7e8202. Latest consensus state: current_term: 1 leader_uuid: "75e95fcb15334d7c925825c8db7e8202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75e95fcb15334d7c925825c8db7e8202" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34713 } } }
I20260809 15:19:11.770933 7467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:11.773890 7479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:19:11.779500 7479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:19:11.789414 7479 catalog_manager.cc:1347] Generated new cluster ID: fa26c08c58194d9b88b2dd75899ac9d8
I20260809 15:19:11.789564 7479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:19:11.824532 7479 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:19:11.824967 7479 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:19:11.837597 7479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75e95fcb15334d7c925825c8db7e8202: Generated new TSK 0
I20260809 15:19:11.838353 7479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:19:11.855175 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7383
W20260809 15:19:11.856333 7450 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37753
W20260809 15:19:11.857052 15406 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7383: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 15:19:11.857203 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 7383
2026-08-09T15:19:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2049 ms)
[----------] 18 tests from AuthTokenIssuingTest (36749 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:19:11 dist-test-slave-8w9v krb5kdc[7488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:19:11 dist-test-slave-8w9v krb5kdc[7488](info): set up 2 sockets
Aug 09 15:19:11 dist-test-slave-8w9v krb5kdc[7488](info): commencing operation
krb5kdc: starting...
W20260809 15:19:13.021430 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:19:13 dist-test-slave-8w9v krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:19:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:19:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:19:13.335206 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:34215
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:34215,127.15.11.189:39659,127.15.11.188:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:19:13.683454 7504 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:19:13.683872 7504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:13.683996 7504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:13.684170 7504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:13.695106 7504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:13.695231 7504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:13.695349 7504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:13.695431 7504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:13.695528 7504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:13.714676 7504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:34215,127.15.11.189:39659,127.15.11.188:43369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:34215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:13.716733 7504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:13.718722 7504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:13.731782 7515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:13.731846 7510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:13.731873 7511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:13.737542 7504 server_base.cc:1034] running on GCE node
I20260809 15:19:13.738338 7504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:13.739763 7504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:13.741011 7504 hybrid_clock.cc:648] HybridClock initialized: now 1786288753740929 us; error 56 us; skew 500 ppm
I20260809 15:19:13.741331 7504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:13 dist-test-slave-8w9v krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:13.744769 7504 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:19:13.746866 7504 webserver.cc:460] Webserver started at http://127.15.11.190:39837/ using document root <none> and password file <none>
I20260809 15:19:13.748055 7504 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:13.748216 7504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:13.748633 7504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:13.752336 7504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6"
format_stamp: "Formatted at 2026-08-09 15:19:13 on dist-test-slave-8w9v"
server_key: "5cd888df91597633a324dee7d5d82731"
server_key_iv: "c737a89331ff954f5fde0a888ae9be89"
server_key_version: "encryptionkey@0"
I20260809 15:19:13.753180 7504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6"
format_stamp: "Formatted at 2026-08-09 15:19:13 on dist-test-slave-8w9v"
server_key: "5cd888df91597633a324dee7d5d82731"
server_key_iv: "c737a89331ff954f5fde0a888ae9be89"
server_key_version: "encryptionkey@0"
I20260809 15:19:13.758762 7504 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260809 15:19:13.768713 7524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:13.770170 7504 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.000s
I20260809 15:19:13.770324 7504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6"
format_stamp: "Formatted at 2026-08-09 15:19:13 on dist-test-slave-8w9v"
server_key: "5cd888df91597633a324dee7d5d82731"
server_key_iv: "c737a89331ff954f5fde0a888ae9be89"
server_key_version: "encryptionkey@0"
I20260809 15:19:13.770525 7504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:13.790629 7504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:13.795137 7504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:13.795357 7504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:13.812947 7504 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:34215
I20260809 15:19:13.812958 7577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:34215 every 8 connection(s)
I20260809 15:19:13.814340 7504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:19:13.815660 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 7504
I20260809 15:19:13.815912 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:19:13.816790 15406 external_mini_cluster.cc:1428] Setting key 76f2a2f5bb735c19890ef4cdfff20d1b
I20260809 15:19:13.819309 7578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:13.829315 7578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:19:13 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.11.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.189@KRBTEST.COM" created.
I20260809 15:19:13.852664 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:13.833247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:34215 (local address 127.0.0.1:57690)
0809 15:19:13.834004 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:13.834051 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:13.836079 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:13.836952 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:13.836958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:13.837659 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:13.837992 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:13.838009 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:13.839588 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:13.839594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:13.840717 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:13.840729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:13.841639 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:13.841646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:13.841854 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:13.842569 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:13.842592 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:13.845383 (+ 2791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:13.848614 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:13.848631 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:13.848644 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:13.849226 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:13.849691 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:13.849696 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:13.849701 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:13.849869 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:13.850698 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:13.850704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:13.850886 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 15:19:13.851092 (+ 206us) client_negotiation.cc:240] Negotiation successful
0809 15:19:13.851468 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":303,"threads_started":1}
I20260809 15:19:13.852705 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:13.834485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.15.11.190:34215)
0809 15:19:13.835174 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:13.835187 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:13.835250 (+ 63us) server_negotiation.cc:407] Connection header received
0809 15:19:13.836499 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:13.836505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:13.836718 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:13.836922 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:13.838206 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:13.839431 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:13.840855 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:13.841399 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:13.841598 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:13.845616 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:13.845624 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:13.845635 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:13.845664 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:13.848475 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:13.849357 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:13.849364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:13.849385 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:13.849565 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:13.849970 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:13.849973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:13.849975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:13.850565 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:13.850723 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:13.851223 (+ 500us) server_negotiation.cc:299] Negotiation successful
0809 15:19:13.851607 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":479,"thread_start_us":316,"threads_started":1}
I20260809 15:19:13.856068 7578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:19:13.857306 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:13.856816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58632)
0809 15:19:13.857012 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:13.857144 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 15:19:13.857748 7526 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
I20260809 15:19:13.859421 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:13.858967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58634)
0809 15:19:13.859142 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:13.859245 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 15:19:13.859632 7578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:39659: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab.
Entry for principal kudu/127.15.11.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab.
I20260809 15:19:13.887450 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.189:39659
--webserver_interface=127.15.11.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab
--principal=kudu/127.15.11.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:34215,127.15.11.189:39659,127.15.11.188:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:19:13.908979 7578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } attempt: 1
I20260809 15:19:13.910195 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:13.909761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58646)
0809 15:19:13.909957 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:13.910058 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 15:19:13.911226 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:13.910881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58656)
0809 15:19:13.911042 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:13.911103 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 15:19:13.911432 7578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:39659: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
I20260809 15:19:13.979660 7578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } attempt: 2
I20260809 15:19:13.980924 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:13.980416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58662)
0809 15:19:13.980675 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:13.980763 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 15:19:13.981997 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:13.981642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58664)
0809 15:19:13.981791 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:13.981874 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 15:19:13.982167 7578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:39659: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
I20260809 15:19:14.073447 7578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } attempt: 3
I20260809 15:19:14.074680 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.074212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58674)
0809 15:19:14.074421 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.074520 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 15:19:14.076076 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.075507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58686)
0809 15:19:14.075746 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.075855 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260809 15:19:14.076333 7578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:39659: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
I20260809 15:19:14.219599 7578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } attempt: 4
I20260809 15:19:14.221019 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.220483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58698)
0809 15:19:14.220794 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.220881 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260809 15:19:14.222143 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.221794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58702)
0809 15:19:14.221942 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.222025 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 15:19:14.222427 7578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:39659: Network error: Client connection negotiation failed: client connection to 127.15.11.189:39659: connect: Connection refused (error 111)
W20260809 15:19:14.234485 7585 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:19:14.234876 7585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:14.235005 7585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:14.235165 7585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:14.242570 7585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:14.242699 7585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:14.242817 7585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:14.242926 7585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:14.243026 7585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:14.252923 7585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:34215,127.15.11.189:39659,127.15.11.188:43369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab
--principal=kudu/127.15.11.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.189:39659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.11.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:14.254317 7585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:14.255795 7585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:14.268473 7592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:14.268651 7591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:14.268530 7594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:14.269299 7585 server_base.cc:1034] running on GCE node
I20260809 15:19:14.269994 7585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:14.271281 7585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:14.272485 7585 hybrid_clock.cc:648] HybridClock initialized: now 1786288754272404 us; error 54 us; skew 500 ppm
I20260809 15:19:14.272761 7585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:14.276062 7585 init.cc:377] Logged in from keytab as kudu/127.15.11.189@KRBTEST.COM (short username kudu)
I20260809 15:19:14.277441 7585 webserver.cc:460] Webserver started at http://127.15.11.189:40167/ using document root <none> and password file <none>
I20260809 15:19:14.278070 7585 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:14.278156 7585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:14.278429 7585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:14.281373 7585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/data/instance:
uuid: "41c2023a5ec74ec3b7c825516666169d"
format_stamp: "Formatted at 2026-08-09 15:19:14 on dist-test-slave-8w9v"
server_key: "6b4e529f145f249453937de670e119f8"
server_key_iv: "06715ae3792efde9101c61977967b8f6"
server_key_version: "encryptionkey@0"
I20260809 15:19:14.282111 7585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/wal/instance:
uuid: "41c2023a5ec74ec3b7c825516666169d"
format_stamp: "Formatted at 2026-08-09 15:19:14 on dist-test-slave-8w9v"
server_key: "6b4e529f145f249453937de670e119f8"
server_key_iv: "06715ae3792efde9101c61977967b8f6"
server_key_version: "encryptionkey@0"
I20260809 15:19:14.287112 7585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 15:19:14.290553 7602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:14.291828 7585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:19:14.291998 7585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/wal
uuid: "41c2023a5ec74ec3b7c825516666169d"
format_stamp: "Formatted at 2026-08-09 15:19:14 on dist-test-slave-8w9v"
server_key: "6b4e529f145f249453937de670e119f8"
server_key_iv: "06715ae3792efde9101c61977967b8f6"
server_key_version: "encryptionkey@0"
I20260809 15:19:14.292265 7585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:14.312036 7585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:14.316550 7585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:14.316840 7585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:14.335457 7585 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.189:39659
I20260809 15:19:14.335469 7664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.189:39659 every 8 connection(s)
I20260809 15:19:14.336889 7585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
I20260809 15:19:14.340674 7665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:14.345546 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 7585
I20260809 15:19:14.345752 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-1/wal/instance
I20260809 15:19:14.346200 15406 external_mini_cluster.cc:1428] Setting key 416478b53e750ebe79b957cc5acb33d2
I20260809 15:19:14.346881 7665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:19:14.367431 7582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.349452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.15.11.190:34215)
0809 15:19:14.349634 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.349639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.350910 (+ 1271us) server_negotiation.cc:407] Connection header received
0809 15:19:14.352797 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.352804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.352899 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.353012 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.354482 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.355173 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.356780 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.357188 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.357324 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.361393 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.361398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.361402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.361435 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.364178 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.365156 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.365160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.365162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.365282 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.365744 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.365747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.365749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.366024 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.366136 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.366996 (+ 860us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.367191 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:19:14.368942 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.350027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:34215 (local address 127.0.0.1:57698)
0809 15:19:14.350753 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.350789 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:14.352523 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:14.353241 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:14.353248 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:14.353733 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:14.354271 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.354287 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.355342 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.355346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.356614 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.356627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.357422 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.357429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.357658 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.358316 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:14.358337 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:14.361168 (+ 2831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:14.364318 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.364329 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.364346 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.365036 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.365388 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.365396 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.365402 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.365631 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.366165 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:14.366177 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:14.366620 (+ 443us) client_negotiation.cc:769] Sending connection context
0809 15:19:14.366916 (+ 296us) client_negotiation.cc:240] Negotiation successful
0809 15:19:14.367440 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":310,"threads_started":1}
I20260809 15:19:14.370684 7665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.11.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.188@KRBTEST.COM" created.
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
I20260809 15:19:14.388294 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.371368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58714)
0809 15:19:14.371569 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.371592 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:14.371705 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:14.372563 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:14.372575 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:14.373108 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:14.373326 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.373342 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.374747 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.374753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.376444 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.376464 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.377291 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.377303 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.377509 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.378095 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:14.378122 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:14.382630 (+ 4508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:14.385820 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.385827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.385831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.386195 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.386570 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.386577 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.386580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.386722 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.387291 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:14.387295 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:14.387432 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 15:19:14.387585 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 15:19:14.387863 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 15:19:14.388408 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.371506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.15.11.189:39659)
0809 15:19:14.372116 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.372168 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.372203 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:19:14.372246 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.372251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.372407 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.372522 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.373516 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.374604 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.376602 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.377127 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.377221 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.382838 (+ 5617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.382849 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.382861 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.382891 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.385675 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.386323 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.386332 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.386337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.386457 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.386821 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.386828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.386831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.387161 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.387382 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.387719 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.387979 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":469,"thread_start_us":308,"threads_started":1}
I20260809 15:19:14.390905 7665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:19:14.391980 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.391559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50144)
0809 15:19:14.391719 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.391827 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 15:19:14.392249 7606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
I20260809 15:19:14.393874 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.393558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50158)
0809 15:19:14.393700 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.393776 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:19:14.394186 7665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:43369: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab.
Entry for principal kudu/127.15.11.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab.
I20260809 15:19:14.418730 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.188:43369
--webserver_interface=127.15.11.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab
--principal=kudu/127.15.11.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:34215,127.15.11.189:39659,127.15.11.188:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:19:14.443502 7665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } attempt: 1
I20260809 15:19:14.444717 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.444241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50162)
0809 15:19:14.444452 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.444559 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 15:19:14.445721 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.445387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50170)
0809 15:19:14.445561 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.445627 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 15:19:14.445976 7665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:43369: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
I20260809 15:19:14.514207 7665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } attempt: 2
I20260809 15:19:14.515435 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.514975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50186)
0809 15:19:14.515173 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.515267 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 15:19:14.516498 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.516196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50202)
0809 15:19:14.516318 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.516383 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 15:19:14.516727 7665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:43369: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
I20260809 15:19:14.521634 7578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } attempt: 5
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
I20260809 15:19:14.535681 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.522235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58722)
0809 15:19:14.522432 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.522457 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:14.522562 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:14.522993 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:14.522997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:14.523348 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:14.523545 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.523555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.525047 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.525051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.525912 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.525925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.526829 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.526837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.527076 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.527704 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:14.527722 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:14.529781 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:14.533524 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.533527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.533530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.533867 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.534265 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.534268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.534270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.534390 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.535007 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:14.535010 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:14.535136 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 15:19:14.535242 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 15:19:14.535448 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 15:19:14.536020 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.522296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58722 (local address 127.15.11.189:39659)
0809 15:19:14.522493 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.522501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.522527 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:19:14.522697 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.522705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.522802 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.522969 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.523720 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.524918 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.526063 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.526662 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.526791 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.529971 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.529977 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.529982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.530018 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.533376 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.534005 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.534010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.534014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.534157 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.534501 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.534509 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.534513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.534897 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.535040 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.535380 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.535610 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:19:14.537026 7578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:19:14.538018 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.537661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50214)
0809 15:19:14.537817 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.537885 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 15:19:14.538898 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.538605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50224)
0809 15:19:14.538728 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.538805 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 15:19:14.539089 7578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:43369: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
I20260809 15:19:14.590382 7578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } attempt: 1
I20260809 15:19:14.591560 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.591088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50232)
0809 15:19:14.591286 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.591393 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 15:19:14.592551 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.592243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50242)
0809 15:19:14.592377 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.592450 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 15:19:14.592756 7578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:43369: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
I20260809 15:19:14.608081 7665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } attempt: 3
I20260809 15:19:14.609247 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.608801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50246)
0809 15:19:14.609014 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.609104 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 15:19:14.610258 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.609862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50260)
0809 15:19:14.610090 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.610159 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260809 15:19:14.610459 7665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:43369: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
I20260809 15:19:14.660948 7578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } attempt: 2
I20260809 15:19:14.662104 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.661619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50264)
0809 15:19:14.661839 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.661944 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 15:19:14.663314 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.662845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50268)
0809 15:19:14.663032 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.663145 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 15:19:14.663573 7578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:43369: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
I20260809 15:19:14.753659 7665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } attempt: 4
I20260809 15:19:14.754854 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.754326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50276)
0809 15:19:14.754601 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.754689 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 15:19:14.756013 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.755641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50284)
0809 15:19:14.755831 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.755903 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260809 15:19:14.756278 7665 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:43369: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
W20260809 15:19:14.767132 7672 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:19:14.767475 7672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:14.767589 7672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:14.767750 7672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 15:19:14.769788 7578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } attempt: 3
I20260809 15:19:14.770767 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.770325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50298)
0809 15:19:14.770532 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.770620 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 15:19:14.771802 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.771430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50304)
0809 15:19:14.771637 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.771704 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 15:19:14.772038 7578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:43369: Network error: Client connection negotiation failed: client connection to 127.15.11.188:43369: connect: Connection refused (error 111)
W20260809 15:19:14.775146 7672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:14.775246 7672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:14.775367 7672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:14.775449 7672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:14.775529 7672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:14.785475 7672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:34215,127.15.11.189:39659,127.15.11.188:43369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab
--principal=kudu/127.15.11.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.188:43369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.11.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:14.786759 7672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:14.788060 7672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:14.800302 7677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:14.800261 7678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:14.800205 7680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:14.801616 7672 server_base.cc:1034] running on GCE node
I20260809 15:19:14.802276 7672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:14.803521 7672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:14.804733 7672 hybrid_clock.cc:648] HybridClock initialized: now 1786288754804651 us; error 50 us; skew 500 ppm
I20260809 15:19:14.805009 7672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:14.808077 7672 init.cc:377] Logged in from keytab as kudu/127.15.11.188@KRBTEST.COM (short username kudu)
I20260809 15:19:14.809587 7672 webserver.cc:460] Webserver started at http://127.15.11.188:34209/ using document root <none> and password file <none>
I20260809 15:19:14.810266 7672 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:14.810382 7672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:14.810716 7672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:14.814494 7672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/data/instance:
uuid: "978bd69ff6f84687b02782da6535ef79"
format_stamp: "Formatted at 2026-08-09 15:19:14 on dist-test-slave-8w9v"
server_key: "08d82f22c939b7442164901b8c47c5a9"
server_key_iv: "cdfc860341db9990b4ea6dd77034bd31"
server_key_version: "encryptionkey@0"
I20260809 15:19:14.817036 7672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/wal/instance:
uuid: "978bd69ff6f84687b02782da6535ef79"
format_stamp: "Formatted at 2026-08-09 15:19:14 on dist-test-slave-8w9v"
server_key: "08d82f22c939b7442164901b8c47c5a9"
server_key_iv: "cdfc860341db9990b4ea6dd77034bd31"
server_key_version: "encryptionkey@0"
I20260809 15:19:14.824427 7672 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260809 15:19:14.827963 7689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:14.829304 7672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:19:14.829496 7672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/wal
uuid: "978bd69ff6f84687b02782da6535ef79"
format_stamp: "Formatted at 2026-08-09 15:19:14 on dist-test-slave-8w9v"
server_key: "08d82f22c939b7442164901b8c47c5a9"
server_key_iv: "cdfc860341db9990b4ea6dd77034bd31"
server_key_version: "encryptionkey@0"
I20260809 15:19:14.829726 7672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:14.845306 7672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:14.849330 7672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:14.849601 7672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:14.868662 7672 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.188:43369
I20260809 15:19:14.868698 7741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.188:43369 every 8 connection(s)
I20260809 15:19:14.870028 7672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
I20260809 15:19:14.873874 7742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:14.877962 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 7672
I20260809 15:19:14.878165 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786288392170773-15406-0/minicluster-data/master-2/wal/instance
I20260809 15:19:14.878615 15406 external_mini_cluster.cc:1428] Setting key 22f20508e3139d6e0b4eba31a66def83
I20260809 15:19:14.881021 7742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:19:14.901012 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.884910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57712 (local address 127.15.11.190:34215)
0809 15:19:14.885449 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.885458 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.886014 (+ 556us) server_negotiation.cc:407] Connection header received
0809 15:19:14.886196 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.886201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.886281 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.886403 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.887579 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.888420 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.890026 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.890816 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.890960 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.894428 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.894435 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.894440 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.894478 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.897850 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.898654 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.898659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.898662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.898820 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.899283 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.899287 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.899290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.899639 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.899800 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.900358 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.900611 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":292,"threads_started":1}
I20260809 15:19:14.901903 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.884922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:34215 (local address 127.0.0.1:57712)
0809 15:19:14.885857 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.885897 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:14.886060 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:14.886458 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:14.886462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:14.887047 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:14.887420 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.887431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.888578 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.888584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.889839 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.889858 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.891034 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.891039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.891166 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.891742 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:14.891762 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:14.894227 (+ 2465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:14.897979 (+ 3752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.897993 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.898003 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.898530 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.898950 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.898955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.898959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.899172 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.899768 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:14.899774 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:14.899943 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 15:19:14.900260 (+ 317us) client_negotiation.cc:240] Negotiation successful
0809 15:19:14.900710 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":42,"thread_start_us":539,"threads_started":1}
I20260809 15:19:14.901921 7746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.880432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.15.11.188:43369)
0809 15:19:14.881303 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.881314 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.881357 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:19:14.882974 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.882997 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.883403 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.883698 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.885580 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.887213 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.888594 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.889259 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.889468 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.894110 (+ 4642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.894118 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.894128 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.894156 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.897833 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.898672 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.898680 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.898683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.898859 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.899289 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.899293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.899295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.899930 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.900237 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.900599 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.900824 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":684,"thread_start_us":398,"threads_started":1}
I20260809 15:19:14.903447 7742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:19:14.917004 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.904254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:39659 (local address 127.0.0.1:58728)
0809 15:19:14.904770 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.904784 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:14.904923 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:14.905295 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:14.905299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:14.905654 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:14.905784 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.905792 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.906723 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.906730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.907582 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.907593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.908342 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.908347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.908558 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.909084 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:14.909100 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:14.911237 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:14.914426 (+ 3189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.914432 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.914435 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.914886 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.915353 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.915356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.915358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.915502 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.916184 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:14.916189 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:14.916311 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 15:19:14.916430 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 15:19:14.916616 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433}
I20260809 15:19:14.917092 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.904347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.15.11.189:39659)
0809 15:19:14.904552 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.904557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.904909 (+ 352us) server_negotiation.cc:407] Connection header received
0809 15:19:14.905070 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.905076 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.905160 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.905272 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.905938 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.906589 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.907728 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.908206 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.908331 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.911423 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.911428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.911431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.911460 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.914288 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.915030 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.915037 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.915043 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.915224 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.915620 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.915625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.915628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.916004 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.916200 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.916558 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.916789 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 15:19:14.918095 7742 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:19:14.919412 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.905042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57716 (local address 127.15.11.190:34215)
0809 15:19:14.905211 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.905217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.905258 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:19:14.905444 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.905449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.905524 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.905625 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.906740 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.907403 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.908740 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.909164 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.909298 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.913453 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.913457 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.913460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.913489 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.916562 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.917338 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.917343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.917346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.917502 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.917903 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.917910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.917913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.918172 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.918439 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.918802 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.919027 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
I20260809 15:19:14.931896 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.918783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50320)
0809 15:19:14.919167 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.919189 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:14.919296 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:14.919723 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:14.919726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:14.920042 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:14.920221 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.920230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.921173 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.921177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.922148 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.922159 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.922836 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.922841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.923082 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.923611 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:14.923632 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:14.926197 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:14.929419 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.929426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.929431 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.929947 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.930282 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.930289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.930293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.930471 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.931076 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:14.931082 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:14.931254 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 15:19:14.931425 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 15:19:14.931625 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":28}
I20260809 15:19:14.932224 7746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.919002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50320 (local address 127.15.11.188:43369)
0809 15:19:14.919174 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.919179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.919258 (+ 79us) server_negotiation.cc:407] Connection header received
0809 15:19:14.919411 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.919416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.919491 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.919587 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.920343 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.921034 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.922284 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.922697 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.922826 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.926395 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.926401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.926409 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.926451 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.929281 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.930058 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.930062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.930064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.930170 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.930583 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.930590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.930596 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.930936 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.931101 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.931556 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.931796 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":60}
I20260809 15:19:14.936431 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.921040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58730 (local address 127.15.11.189:39659)
0809 15:19:14.921199 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.921206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.921225 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:19:14.921292 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.921297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.921359 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.921457 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.922531 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.923202 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.925049 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.925687 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.925787 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.930456 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.930461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.930463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.930491 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.933777 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.934517 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.934523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.934527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.934683 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.935042 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.935047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.935050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.935369 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.935643 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.935831 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.936082 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:19:14.938436 7742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79: Bootstrap starting.
I20260809 15:19:14.941177 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:19:14.942126 7742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:14.943171 7742 log.cc:824] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:14.945807 7742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79: No bootstrap required, opened a new log
I20260809 15:19:14.943725 7553 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57716
I20260809 15:19:14.949258 7578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } attempt: 4
I20260809 15:19:14.955010 7742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } }
I20260809 15:19:14.955390 7742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 978bd69ff6f84687b02782da6535ef79, State: Initialized, Role: FOLLOWER
I20260809 15:19:14.955881 7742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } }
Aug 09 15:19:14 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288753, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
I20260809 15:19:14.956950 7750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:14.957350 7750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:14.958107 7742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:14.964389 7581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.949880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50330)
0809 15:19:14.950073 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:14.950098 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:14.950206 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:14.950531 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:14.950535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:14.951006 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:14.951179 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.951189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.952280 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.952285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.953182 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:14.953194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.954010 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.954016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:14.954217 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.954842 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:14.954860 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:14.957420 (+ 2560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:14.961396 (+ 3976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.961403 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.961407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.961955 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.962402 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:14.962407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:14.962411 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:14.962564 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:14.963335 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:14.963341 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:14.963515 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 15:19:14.963660 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 15:19:14.963960 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 15:19:14.964429 7746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:14.949971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50330 (local address 127.15.11.188:43369)
0809 15:19:14.950169 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:14.950174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:14.950193 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:19:14.950304 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:14.950314 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:14.950386 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:14.950496 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:14.951327 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.952097 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.953337 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:14.953834 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:14.954000 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:14.957574 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:14.957580 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:14.957587 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:14.957639 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:14.961265 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.962065 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.962081 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.962087 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.962275 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:14.962680 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:14.962686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:14.962689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:14.963205 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:14.963364 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:14.963769 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 15:19:14.964007 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 15:19:14.969141 7764 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:19:14.969264 7764 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:19:14.971246 7578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6: Bootstrap starting.
I20260809 15:19:14.975235 7578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:14.966627 7672 master_runner.cc:186] Error getting master registration for 127.15.11.190:34215: OK, instance_id { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" instance_seqno: 1786288753808017 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:19:14.975540 7672 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:19:14.976624 7578 log.cc:824] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:14.979786 7578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6: No bootstrap required, opened a new log
I20260809 15:19:14.987272 7578 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } }
I20260809 15:19:14.987661 7578 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d4c8d65aa4416b8f89b30fb72fabe6, State: Initialized, Role: FOLLOWER
I20260809 15:19:14.988199 7578 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } }
I20260809 15:19:14.989223 7767 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:14.989614 7767 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:14.990408 7578 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:14.990463 7553 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:19:14.990674 7553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:14.993559 7553 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } }
I20260809 15:19:14.995862 7716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "978bd69ff6f84687b02782da6535ef79"
I20260809 15:19:14.996424 7716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:15.000039 7716 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d4c8d65aa4416b8f89b30fb72fabe6 in term 1.
I20260809 15:19:15.000437 7553 leader_election.cc:288] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [CANDIDATE]: Term 1 election: Requested vote from peers 41c2023a5ec74ec3b7c825516666169d (127.15.11.189:39659), 978bd69ff6f84687b02782da6535ef79 (127.15.11.188:43369)
I20260809 15:19:15.000738 7529 leader_election.cc:302] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 978bd69ff6f84687b02782da6535ef79, b1d4c8d65aa4416b8f89b30fb72fabe6; no voters:
I20260809 15:19:15.001210 7767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:19:15.014648 7767 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [term 1 LEADER]: Becoming Leader. State: Replica: b1d4c8d65aa4416b8f89b30fb72fabe6, State: Running, Role: LEADER
I20260809 15:19:15.016234 7767 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } }
I20260809 15:19:15.033238 7774 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1d4c8d65aa4416b8f89b30fb72fabe6. Latest consensus state: current_term: 1 leader_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:15.033715 7774 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:15.043687 7785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:19:15.046802 7788 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 15:19:15.048306 7531 tablet.cc:2369] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:19:15.055477 7665 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } attempt: 5
I20260809 15:19:15.060853 7785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 15:19:15 dist-test-slave-8w9v krb5kdc[7488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288754, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
I20260809 15:19:15.074986 7746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:15.056733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50336 (local address 127.15.11.188:43369)
0809 15:19:15.056954 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.056961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.056978 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:19:15.057035 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.057040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.057148 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.057256 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.058399 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.059439 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.061284 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.062023 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.062147 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:15.066682 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:15.066688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:15.066694 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:15.066732 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:15.070549 (+ 3817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:15.071705 (+ 1156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:15.071724 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:15.071728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:15.071897 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:15.072559 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:15.072564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:15.072567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:15.072924 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:15.073062 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:15.073911 (+ 849us) server_negotiation.cc:299] Negotiation successful
0809 15:19:15.074305 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 15:19:15.051283 7504 master_runner.cc:186] Error getting master registration for 127.15.11.189:39659: OK, instance_id { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" instance_seqno: 1786288754330104 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:19:15.076522 7504 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:19:15.076617 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:15.056085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:43369 (local address 127.0.0.1:50336)
0809 15:19:15.056422 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:15.056441 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:15.056528 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:15.057499 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:15.057504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:15.058017 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:15.058220 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:15.058233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.059610 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.059615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:15.061035 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:15.061050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.062181 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.062188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:15.062413 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:15.063071 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:15.063099 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:15.066404 (+ 3305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:15.070738 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:15.070743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:15.070747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:15.071522 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:15.072092 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:15.072097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:15.072101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:15.072366 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:15.073186 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:15.073191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:15.073430 (+ 239us) client_negotiation.cc:769] Sending connection context
0809 15:19:15.075568 (+ 2138us) client_negotiation.cc:240] Negotiation successful
0809 15:19:15.076031 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260809 15:19:15.085742 7665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d: Bootstrap starting.
I20260809 15:19:15.087844 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:15.035386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58744 (local address 127.15.11.189:39659)
0809 15:19:15.035630 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.035639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.047695 (+ 12056us) server_negotiation.cc:407] Connection header received
0809 15:19:15.048523 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.048529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.048633 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.048712 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.053894 (+ 5182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.055285 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.065362 (+ 10077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.066233 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.066355 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:15.080306 (+ 13951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:15.080311 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:15.080315 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:15.080348 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:15.083843 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:15.084730 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:15.084735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:15.084738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:15.084897 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:15.085373 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:15.085378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:15.085381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:15.085748 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:15.085995 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:15.087161 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0809 15:19:15.087407 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 15:19:15.092943 7665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:15.094307 7665 log.cc:824] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d: Log is configured to *not* fsync() on all Append() calls
W20260809 15:19:15.101764 7695 tablet.cc:2369] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:19:15.104336 7716 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [term 1 FOLLOWER]: Refusing update from remote peer b1d4c8d65aa4416b8f89b30fb72fabe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:19:15.105386 7767 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:19:15.111986 7746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:15.062870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50346 (local address 127.15.11.188:43369)
0809 15:19:15.075502 (+ 12632us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.075508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.086471 (+ 10963us) server_negotiation.cc:407] Connection header received
0809 15:19:15.089890 (+ 3419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.089899 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.090022 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.090128 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.096256 (+ 6128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.097366 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.100083 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.100782 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.100879 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:15.104361 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:15.104368 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:15.104372 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:15.104409 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:15.108661 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:15.109503 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:15.109510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:15.109513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:15.109687 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:15.110160 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:15.110166 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:15.110171 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:15.110506 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:15.110745 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:15.111296 (+ 551us) server_negotiation.cc:299] Negotiation successful
0809 15:19:15.111535 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12429,"thread_start_us":31955,"threads_started":1}
I20260809 15:19:15.118927 7750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1d4c8d65aa4416b8f89b30fb72fabe6. Latest consensus state: current_term: 1 leader_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:15.119212 7750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:15.121608 7774 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:15.121848 7774 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:15.122704 7796 mvcc.cc:205] Tried to move back new op lower bound from 7316638740872953856 to 7316638740564340736. Current Snapshot: MvccSnapshot[applied={T|T < 7316638740872953856}]
I20260809 15:19:15.124397 7747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:15.042999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57730 (local address 127.15.11.190:34215)
0809 15:19:15.043186 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.043193 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.072483 (+ 29290us) server_negotiation.cc:407] Connection header received
0809 15:19:15.073312 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.073318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.073406 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.073478 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.079713 (+ 6235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.080861 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.086852 (+ 5991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:15.087564 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:15.087662 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:15.097558 (+ 9896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:15.097565 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:15.097570 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:15.097632 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:15.101692 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:15.103433 (+ 1741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:15.103439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:15.103442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:15.110756 (+ 7314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:15.122423 (+ 11667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:15.122428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:15.122432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:15.122847 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:15.123128 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:15.123721 (+ 593us) server_negotiation.cc:299] Negotiation successful
0809 15:19:15.123979 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 15:19:15.128654 7665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d: No bootstrap required, opened a new log
I20260809 15:19:15.130403 7750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:15.130672 7750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:15.136586 7785 catalog_manager.cc:1347] Generated new cluster ID: 81e22ae1f2484aaa8e7c37ac806a190b
I20260809 15:19:15.136682 7785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:19:15.140336 7665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } }
I20260809 15:19:15.140729 7665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41c2023a5ec74ec3b7c825516666169d, State: Initialized, Role: FOLLOWER
I20260809 15:19:15.141242 7665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } }
I20260809 15:19:15.142418 7801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:15.143126 7801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:15.143498 7665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:15.143841 7632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:15.147248 7632 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [term 1 FOLLOWER]: Refusing update from remote peer b1d4c8d65aa4416b8f89b30fb72fabe6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:19:15.147730 7633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "41c2023a5ec74ec3b7c825516666169d"
I20260809 15:19:15.148042 7767 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:19:15.151197 7633 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d4c8d65aa4416b8f89b30fb72fabe6 in term 1.
I20260809 15:19:15.178869 7785 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:19:15.182904 7785 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:19:15.202678 7801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1d4c8d65aa4416b8f89b30fb72fabe6. Latest consensus state: current_term: 1 leader_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:15.203070 7801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:15.202764 7808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:15.203311 7808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:15.205983 7774 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1d4c8d65aa4416b8f89b30fb72fabe6" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 34215 } } peers { permanent_uuid: "41c2023a5ec74ec3b7c825516666169d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 39659 } } peers { permanent_uuid: "978bd69ff6f84687b02782da6535ef79" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 43369 } } }
I20260809 15:19:15.206236 7774 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6 [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:15.215041 7785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1d4c8d65aa4416b8f89b30fb72fabe6: Generated new TSK 0
I20260809 15:19:15.216212 7785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:19:15.221297 7817 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 15:19:15.311379 7747 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.305251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57740 (local address 127.15.11.190:34215)
0809 15:19:15.305506 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.305515 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.309234 (+ 3719us) server_negotiation.cc:407] Connection header received
0809 15:19:15.309728 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.309735 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.309821 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.309898 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.311089 (+ 1191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57740: BlockingRecv error: recv got EOF from 127.0.0.1:57740 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 15:19:15.313831 7746 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.309379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.15.11.188:43369)
0809 15:19:15.309562 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.309568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.312111 (+ 2543us) server_negotiation.cc:407] Connection header received
0809 15:19:15.312414 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.312420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.312508 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.312577 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.313607 (+ 1030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50356: BlockingRecv error: recv got EOF from 127.0.0.1:50356 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 15:19:15.315940 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.304702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.15.11.189:39659)
0809 15:19:15.304956 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.304965 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.308830 (+ 3865us) server_negotiation.cc:407] Connection header received
0809 15:19:15.314534 (+ 5704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.314542 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.314636 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.314781 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.315715 (+ 934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58748: BlockingRecv error: recv got EOF from 127.0.0.1:58748 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 15:19:15.322397 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.320524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.15.11.189:39659)
0809 15:19:15.320683 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.320688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.320988 (+ 300us) server_negotiation.cc:407] Connection header received
0809 15:19:15.321202 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.321207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.321337 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.321450 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.322159 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58756: BlockingRecv error: recv got EOF from 127.0.0.1:58756 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 15:19:15.338127 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.334346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.15.11.189:39659)
0809 15:19:15.334534 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.334540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.336495 (+ 1955us) server_negotiation.cc:407] Connection header received
0809 15:19:15.336789 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.336796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.336905 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.336985 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.337924 (+ 939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58762: BlockingRecv error: recv got EOF from 127.0.0.1:58762 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 15:19:15.370698 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.368424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58776 (local address 127.15.11.189:39659)
0809 15:19:15.368603 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.368611 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.369154 (+ 543us) server_negotiation.cc:407] Connection header received
0809 15:19:15.369367 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.369373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.369462 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.369571 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.370464 (+ 893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58776: BlockingRecv error: recv got EOF from 127.0.0.1:58776 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 15:19:15.383876 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.381636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.15.11.189:39659)
0809 15:19:15.381841 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.381849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.382389 (+ 540us) server_negotiation.cc:407] Connection header received
0809 15:19:15.382595 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.382601 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.382685 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.382794 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.383658 (+ 864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58792: BlockingRecv error: recv got EOF from 127.0.0.1:58792 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260809 15:19:15.398190 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.395389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58802 (local address 127.15.11.189:39659)
0809 15:19:15.395567 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.395572 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.396201 (+ 629us) server_negotiation.cc:407] Connection header received
0809 15:19:15.396440 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.396445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.396541 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.396647 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.397972 (+ 1325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58802: BlockingRecv error: recv got EOF from 127.0.0.1:58802 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 15:19:15.419924 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.417590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.15.11.189:39659)
0809 15:19:15.417780 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.417786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.418370 (+ 584us) server_negotiation.cc:407] Connection header received
0809 15:19:15.418577 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.418582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.418682 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.418792 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.419680 (+ 888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58810: BlockingRecv error: recv got EOF from 127.0.0.1:58810 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 15:19:15.460906 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.458126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58824 (local address 127.15.11.189:39659)
0809 15:19:15.458380 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.458387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.459059 (+ 672us) server_negotiation.cc:407] Connection header received
0809 15:19:15.459324 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.459332 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.459433 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.459544 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.460620 (+ 1076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58824: BlockingRecv error: recv got EOF from 127.0.0.1:58824 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
W20260809 15:19:15.530547 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.528440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.15.11.189:39659)
0809 15:19:15.528675 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.528681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.529044 (+ 363us) server_negotiation.cc:407] Connection header received
0809 15:19:15.529289 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.529302 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.529397 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.529518 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.530304 (+ 786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58840: BlockingRecv error: recv got EOF from 127.0.0.1:58840 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
W20260809 15:19:15.664103 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.662244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.15.11.189:39659)
0809 15:19:15.662510 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.662518 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.662754 (+ 236us) server_negotiation.cc:407] Connection header received
0809 15:19:15.662931 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.662935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.663021 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.663111 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.663848 (+ 737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58846: BlockingRecv error: recv got EOF from 127.0.0.1:58846 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260809 15:19:15.926249 7669 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:15.924284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.15.11.189:39659)
0809 15:19:15.924555 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:15.924562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:15.924815 (+ 253us) server_negotiation.cc:407] Connection header received
0809 15:19:15.925078 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:15.925082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:15.925152 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:15.925262 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:15.926035 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58848: BlockingRecv error: recv got EOF from 127.0.0.1:58848 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 15:19:15.971266 7764 catalog_manager.cc:1259] Loaded cluster ID: 81e22ae1f2484aaa8e7c37ac806a190b
I20260809 15:19:15.971362 7764 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79: loading cluster ID for follower catalog manager: success
I20260809 15:19:15.973795 7764 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79: acquiring CA information for follower catalog manager: success
I20260809 15:19:15.975641 7764 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 978bd69ff6f84687b02782da6535ef79: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:19:16.225796 7817 catalog_manager.cc:1259] Loaded cluster ID: 81e22ae1f2484aaa8e7c37ac806a190b
I20260809 15:19:16.225997 7817 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d: loading cluster ID for follower catalog manager: success
I20260809 15:19:16.228605 7817 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d: acquiring CA information for follower catalog manager: success
I20260809 15:19:16.230400 7817 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 41c2023a5ec74ec3b7c825516666169d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:19:16.453027 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:16.445488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.15.11.189:39659)
0809 15:19:16.446196 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:16.446203 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:16.446524 (+ 321us) server_negotiation.cc:407] Connection header received
0809 15:19:16.446753 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:16.446757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:16.446906 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:16.447035 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:16.447608 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.448529 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.450516 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.450946 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.451036 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:16.451672 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:16.452261 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:16.452395 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:16.452546 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 15:19:16.452610 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":327,"threads_started":1}
I20260809 15:19:16.469877 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:16.461241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50362 (local address 127.15.11.188:43369)
0809 15:19:16.461845 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:16.461852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:16.464581 (+ 2729us) server_negotiation.cc:407] Connection header received
0809 15:19:16.464829 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:16.464833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:16.465005 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:16.465175 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:16.465593 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.466450 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.467692 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.468089 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.468287 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:16.468783 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:16.469281 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:16.469452 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:16.469510 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 15:19:16.469599 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":333,"threads_started":1}
I20260809 15:19:16.493471 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:16.476343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.15.11.189:39659)
0809 15:19:16.476520 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:16.476526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:16.483953 (+ 7427us) server_negotiation.cc:407] Connection header received
0809 15:19:16.484461 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:16.484471 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:16.484617 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:16.484697 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:16.486302 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.487800 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.490752 (+ 2952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.491721 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.491834 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:16.492555 (+ 721us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:16.492695 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:16.492763 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:16.493043 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 15:19:16.493140 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 15:19:16.498929 7898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:16.483125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50370 (local address 127.15.11.188:43369)
0809 15:19:16.483273 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:16.483278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:16.491685 (+ 8407us) server_negotiation.cc:407] Connection header received
0809 15:19:16.492101 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:16.492107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:16.492238 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:16.492307 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:16.493086 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.494123 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.497085 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.497635 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.497776 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:16.498163 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:16.498268 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:16.498370 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:16.498527 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 15:19:16.498605 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 15:19:16.499507 7905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:16.475959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57744 (local address 127.15.11.190:34215)
0809 15:19:16.492093 (+ 16134us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:16.492099 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:16.492135 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:19:16.492256 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:16.492260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:16.492504 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:16.493351 (+ 847us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:16.493551 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.494436 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.496648 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:16.497236 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:16.497359 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:16.497981 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:16.498470 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:16.498585 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:16.499084 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 15:19:16.499181 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15998,"thread_start_us":16670,"threads_started":1}
I20260809 15:19:16.504711 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7504
I20260809 15:19:16.505993 7573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:19:16.616068 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 7504
I20260809 15:19:16.642328 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7585
I20260809 15:19:16.643499 7660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:19:16.749905 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 7585
I20260809 15:19:16.774673 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7672
I20260809 15:19:16.775839 7737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:19:16.882448 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 7672
2026-08-09T15:19:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5031 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:19:17 dist-test-slave-8w9v krb5kdc[7914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:19:17 dist-test-slave-8w9v krb5kdc[7914](info): set up 2 sockets
Aug 09 15:19:17 dist-test-slave-8w9v krb5kdc[7914](info): commencing operation
krb5kdc: starting...
W20260809 15:19:18.061815 15406 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.005s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/test-user.keytab.
Aug 09 15:19:18 dist-test-slave-8w9v krb5kdc[7914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:19:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:19:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
Entry for principal kudu/127.15.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab.
I20260809 15:19:18.359673 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:42381
--webserver_interface=127.15.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:42381,127.15.11.189:42337,127.15.11.188:36695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:19:18.697007 7928 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:19:18.697340 7928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:18.697497 7928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:18.697621 7928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:18.706065 7928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:18.706166 7928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:18.706307 7928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:18.706399 7928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:18.706511 7928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:18.716986 7928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:42381,127.15.11.189:42337,127.15.11.188:36695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.190.keytab
--principal=kudu/127.15.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.190:42381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:18.718329 7928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:18.719772 7928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:18.732012 7935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:18.731926 7938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:18.731976 7936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:18.732874 7928 server_base.cc:1034] running on GCE node
I20260809 15:19:18.733841 7928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:18.735321 7928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:18.736591 7928 hybrid_clock.cc:648] HybridClock initialized: now 1786288758736517 us; error 51 us; skew 500 ppm
I20260809 15:19:18.736937 7928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:18 dist-test-slave-8w9v krb5kdc[7914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288758, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:18.740218 7928 init.cc:377] Logged in from keytab as kudu/127.15.11.190@KRBTEST.COM (short username kudu)
I20260809 15:19:18.741715 7928 webserver.cc:460] Webserver started at http://127.15.11.190:42003/ using document root <none> and password file <none>
I20260809 15:19:18.742413 7928 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:18.742544 7928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:18.742865 7928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:18.745911 7928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/data/instance:
uuid: "8f93b17fa5f34ef6bf3a30074508b2ad"
format_stamp: "Formatted at 2026-08-09 15:19:18 on dist-test-slave-8w9v"
server_key: "443eab848b75213fa27857536142f3e8"
server_key_iv: "6536c7a1d45e1c70e0f48b793a0dbab5"
server_key_version: "encryptionkey@0"
I20260809 15:19:18.746701 7928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/wal/instance:
uuid: "8f93b17fa5f34ef6bf3a30074508b2ad"
format_stamp: "Formatted at 2026-08-09 15:19:18 on dist-test-slave-8w9v"
server_key: "443eab848b75213fa27857536142f3e8"
server_key_iv: "6536c7a1d45e1c70e0f48b793a0dbab5"
server_key_version: "encryptionkey@0"
I20260809 15:19:18.751786 7928 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 15:19:18.755249 7945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:18.756659 7928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 15:19:18.756850 7928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/wal
uuid: "8f93b17fa5f34ef6bf3a30074508b2ad"
format_stamp: "Formatted at 2026-08-09 15:19:18 on dist-test-slave-8w9v"
server_key: "443eab848b75213fa27857536142f3e8"
server_key_iv: "6536c7a1d45e1c70e0f48b793a0dbab5"
server_key_version: "encryptionkey@0"
I20260809 15:19:18.757064 7928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:18.782517 7928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:18.787057 7928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:18.787357 7928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:18.806733 7928 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.190:42381
I20260809 15:19:18.806767 8000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.190:42381 every 8 connection(s)
I20260809 15:19:18.808079 7928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/data/info.pb
I20260809 15:19:18.811733 8001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:18.818131 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 7928
I20260809 15:19:18.818310 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-0/wal/instance
I20260809 15:19:18.818758 15406 external_mini_cluster.cc:1428] Setting key 6e1481aea15f0b1588527d794b68d9c2
I20260809 15:19:18.818024 8001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:19:18 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288758, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:19:18.842921 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:18.821291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42381 (local address 127.0.0.1:57382)
0809 15:19:18.822447 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:18.822484 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:18.824190 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:18.824990 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:18.824995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:18.825729 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:18.826060 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:18.826077 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:18.827904 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:18.827912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:18.829768 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:18.829787 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:18.831001 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:18.831010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:18.831261 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:18.832023 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:18.832056 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:18.834260 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:18.838935 (+ 4675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:18.838945 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:18.838954 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:18.839321 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:18.839841 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:18.839845 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:18.839847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:18.840004 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:18.841043 (+ 1039us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:18.841049 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:18.841268 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 15:19:18.841498 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 15:19:18.841876 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":970,"spinlock_wait_cycles":15616,"thread_start_us":805,"threads_started":1}
I20260809 15:19:18.842955 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:18.821276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.15.11.190:42381)
0809 15:19:18.822031 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:18.822041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:18.822603 (+ 562us) server_negotiation.cc:407] Connection header received
0809 15:19:18.824553 (+ 1950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:18.824558 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:18.824772 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:18.824981 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:18.826273 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:18.827722 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:18.829921 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:18.830751 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:18.830933 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:18.834527 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:18.834540 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:18.834558 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:18.834605 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:18.838766 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:18.839498 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:18.839509 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:18.839514 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:18.839727 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:18.840145 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:18.840152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:18.840159 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:18.840897 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:18.841166 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:18.841688 (+ 522us) server_negotiation.cc:299] Negotiation successful
0809 15:19:18.842085 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":574,"thread_start_us":368,"threads_started":1}
I20260809 15:19:18.846058 8001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:19:18.847239 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:18.846744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50820)
0809 15:19:18.846952 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:18.847087 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 15:19:18.847503 7947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
I20260809 15:19:18.849256 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:18.848783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50824)
0809 15:19:18.849010 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:18.849096 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 15:19:18.849489 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:42337: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.15.11.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab.
Entry for principal kudu/127.15.11.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab.
I20260809 15:19:18.890635 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.189:42337
--webserver_interface=127.15.11.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab
--principal=kudu/127.15.11.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:42381,127.15.11.189:42337,127.15.11.188:36695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:19:18.898874 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } attempt: 1
I20260809 15:19:18.900204 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:18.899658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50836)
0809 15:19:18.899869 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:18.899997 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 15:19:18.901242 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:18.900878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50848)
0809 15:19:18.901028 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:18.901101 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 15:19:18.901446 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:42337: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
I20260809 15:19:18.969674 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } attempt: 2
I20260809 15:19:18.970913 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:18.970438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50860)
0809 15:19:18.970670 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:18.970764 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 15:19:18.971936 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:18.971577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50874)
0809 15:19:18.971724 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:18.971809 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:19:18.972105 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:42337: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
I20260809 15:19:19.063450 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } attempt: 3
I20260809 15:19:19.064702 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.064256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50878)
0809 15:19:19.064463 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.064560 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 15:19:19.065781 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.065397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50890)
0809 15:19:19.065563 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.065634 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 15:19:19.065968 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:42337: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
I20260809 15:19:19.209196 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } attempt: 4
I20260809 15:19:19.210577 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.209976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50906)
0809 15:19:19.210337 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.210442 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177}
I20260809 15:19:19.211619 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.211253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50914)
0809 15:19:19.211409 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.211498 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 15:19:19.211813 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.189:42337: Network error: Client connection negotiation failed: client connection to 127.15.11.189:42337: connect: Connection refused (error 111)
W20260809 15:19:19.230742 8009 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:19:19.231097 8009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:19.231223 8009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:19.231359 8009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:19:19.238646 8009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:19.238770 8009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:19.238910 8009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:19.239001 8009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:19.239117 8009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:19.248983 8009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:42381,127.15.11.189:42337,127.15.11.188:36695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.189.keytab
--principal=kudu/127.15.11.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.189:42337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.11.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:19.250332 8009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:19.251734 8009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:19.263990 8014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:19.263957 8015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:19.264587 8017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:19.265175 8009 server_base.cc:1034] running on GCE node
I20260809 15:19:19.266010 8009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:19.267495 8009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:19.268750 8009 hybrid_clock.cc:648] HybridClock initialized: now 1786288759268691 us; error 52 us; skew 500 ppm
I20260809 15:19:19.269095 8009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:19.272482 8009 init.cc:377] Logged in from keytab as kudu/127.15.11.189@KRBTEST.COM (short username kudu)
I20260809 15:19:19.273900 8009 webserver.cc:460] Webserver started at http://127.15.11.189:33619/ using document root <none> and password file <none>
I20260809 15:19:19.274608 8009 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:19.274744 8009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:19.275084 8009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:19.278126 8009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/data/instance:
uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d"
format_stamp: "Formatted at 2026-08-09 15:19:19 on dist-test-slave-8w9v"
server_key: "0a5047a4d5e9699e487ff3fab8469b36"
server_key_iv: "f6461c9d0e3904ba4a1e571a6df07921"
server_key_version: "encryptionkey@0"
I20260809 15:19:19.278924 8009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/wal/instance:
uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d"
format_stamp: "Formatted at 2026-08-09 15:19:19 on dist-test-slave-8w9v"
server_key: "0a5047a4d5e9699e487ff3fab8469b36"
server_key_iv: "f6461c9d0e3904ba4a1e571a6df07921"
server_key_version: "encryptionkey@0"
I20260809 15:19:19.284114 8009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 15:19:19.287488 8026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:19.288844 8009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:19:19.289022 8009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/wal
uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d"
format_stamp: "Formatted at 2026-08-09 15:19:19 on dist-test-slave-8w9v"
server_key: "0a5047a4d5e9699e487ff3fab8469b36"
server_key_iv: "f6461c9d0e3904ba4a1e571a6df07921"
server_key_version: "encryptionkey@0"
I20260809 15:19:19.289268 8009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:19.307331 8009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:19.311245 8009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:19.311514 8009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:19.330123 8009 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.189:42337
I20260809 15:19:19.330130 8081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.189:42337 every 8 connection(s)
I20260809 15:19:19.331494 8009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/data/info.pb
I20260809 15:19:19.335312 8082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:19.340891 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 8009
I20260809 15:19:19.341095 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-1/wal/instance
I20260809 15:19:19.341566 15406 external_mini_cluster.cc:1428] Setting key 207a6d8effc343b46255d9d0926cb11c
I20260809 15:19:19.342141 8082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:19:19.362332 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.344697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57392 (local address 127.15.11.190:42381)
0809 15:19:19.345215 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.345219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.346214 (+ 995us) server_negotiation.cc:407] Connection header received
0809 15:19:19.348072 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.348078 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.348198 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.348321 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.349851 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.350583 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.352017 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.352490 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.352631 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.356167 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.356174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.356185 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.356222 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.358903 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.359621 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.359627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.359631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.359758 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.360244 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.360251 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.360258 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.360596 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.360771 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.361556 (+ 785us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.361828 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":293,"threads_started":1}
I20260809 15:19:19.362967 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.345326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42381 (local address 127.0.0.1:57392)
0809 15:19:19.346040 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.346089 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:19.347801 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:19.348582 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:19.348590 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:19.349100 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:19.349635 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.349652 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.350760 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.350764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.351871 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.351885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.352716 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.352723 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.352971 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.353693 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:19.353714 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:19.355892 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:19.359046 (+ 3154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.359055 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.359069 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.359502 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.359878 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.359883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.359886 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.360059 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.360751 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:19.360759 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:19.361097 (+ 338us) client_negotiation.cc:769] Sending connection context
0809 15:19:19.361450 (+ 353us) client_negotiation.cc:240] Negotiation successful
0809 15:19:19.361838 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":302,"threads_started":1}
I20260809 15:19:19.364765 8082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.11.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.11.188@KRBTEST.COM" created.
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
I20260809 15:19:19.382512 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.365757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50922 (local address 127.15.11.189:42337)
0809 15:19:19.366441 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.366446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.366469 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:19:19.366508 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.366513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.366665 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.366819 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.367540 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.368627 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.369703 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.370297 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.370442 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.376458 (+ 6016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.376469 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.376482 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.376510 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.379274 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.380113 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.380203 (+ 90us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.380213 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.380361 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.380824 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.380828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.380831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.381210 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.381409 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.381843 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.382155 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":514,"thread_start_us":390,"threads_started":1}
I20260809 15:19:19.382540 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.365445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50922)
0809 15:19:19.365656 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.365670 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:19.365749 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:19.366804 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:19.366809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:19.367238 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:19.367364 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.367374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.368753 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.368760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.369560 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.369571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.370447 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.370452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.370630 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.371211 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:19.371227 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:19.376192 (+ 4965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:19.379419 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.379426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.379433 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.379952 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.380495 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.380502 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.380508 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.380703 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.381359 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:19.381373 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:19.381541 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 15:19:19.381699 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 15:19:19.382028 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260809 15:19:19.385082 8082 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:19:19.386365 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.385916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50192)
0809 15:19:19.386079 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.386204 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 15:19:19.386667 8030 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
I20260809 15:19:19.388465 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.388043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50200)
0809 15:19:19.388280 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.388364 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 15:19:19.388726 8082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:36695: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.11.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab.
Entry for principal kudu/127.15.11.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab.
I20260809 15:19:19.411087 15406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskFizLz9/build/asan/bin/kudu
/tmp/dist-test-taskFizLz9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.188:36695
--webserver_interface=127.15.11.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab
--principal=kudu/127.15.11.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.11.148:34329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.11.190:42381,127.15.11.189:42337,127.15.11.188:36695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:19:19.438054 8082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } attempt: 1
I20260809 15:19:19.439476 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.438829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50210)
0809 15:19:19.439107 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.439263 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 15:19:19.440702 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.440239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50222)
0809 15:19:19.440434 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.440542 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 15:19:19.440979 8082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:36695: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
I20260809 15:19:19.509197 8082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } attempt: 2
I20260809 15:19:19.510448 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.509874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50232)
0809 15:19:19.510124 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.510246 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 15:19:19.511053 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } attempt: 5
I20260809 15:19:19.511628 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.511188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50246)
0809 15:19:19.511369 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.511453 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 15:19:19.512001 8082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:36695: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288758, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
I20260809 15:19:19.523638 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.511686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50924 (local address 127.15.11.189:42337)
0809 15:19:19.511841 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.511846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.511995 (+ 149us) server_negotiation.cc:407] Connection header received
0809 15:19:19.512219 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.512223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.512286 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.512387 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.513156 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.513834 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.514954 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.515414 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.515547 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.518620 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.518632 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.518635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.518666 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.521184 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.521806 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.521812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.521815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.521947 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.522333 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.522340 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.522343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.522607 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.522756 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.523136 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.523339 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:19:19.523660 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.511638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50924)
0809 15:19:19.511896 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.511914 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:19.512043 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:19.512431 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:19.512437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:19.512809 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:19.512987 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.512996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.513972 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.513978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.514804 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.514816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.515583 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.515589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.515810 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.516504 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:19.516522 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:19.518440 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:19.521317 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.521323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.521327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.521669 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.522063 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.522069 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.522073 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.522223 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.522726 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:19.522731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:19.522884 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 15:19:19.523020 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 15:19:19.523334 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260809 15:19:19.524595 8001 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:19:19.525576 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.525194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50258)
0809 15:19:19.525354 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.525452 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 15:19:19.526393 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.526092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50270)
0809 15:19:19.526224 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.526294 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 15:19:19.526608 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:36695: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
I20260809 15:19:19.577816 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } attempt: 1
I20260809 15:19:19.579197 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.578498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50274)
0809 15:19:19.578776 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.578932 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 15:19:19.580591 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.580087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50288)
0809 15:19:19.580310 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.580423 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 15:19:19.580921 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:36695: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
I20260809 15:19:19.603221 8082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } attempt: 3
I20260809 15:19:19.604418 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.603902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50294)
0809 15:19:19.604104 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.604245 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 15:19:19.605605 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.605141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50306)
0809 15:19:19.605411 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.605486 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
W20260809 15:19:19.606009 8082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:36695: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
I20260809 15:19:19.649180 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } attempt: 2
I20260809 15:19:19.650518 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.649897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50314)
0809 15:19:19.650173 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.650297 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 15:19:19.651646 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.651196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50330)
0809 15:19:19.651388 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.651480 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 15:19:19.651883 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:36695: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
I20260809 15:19:19.749228 8082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } attempt: 4
I20260809 15:19:19.750702 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.750060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50332)
0809 15:19:19.750395 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.750500 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260809 15:19:19.751998 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.751671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50346)
0809 15:19:19.751835 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.751903 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 15:19:19.752266 8082 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:36695: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
W20260809 15:19:19.755008 8090 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:19:19.755314 8090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:19:19.755393 8090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:19:19.755472 8090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 15:19:19.758109 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } attempt: 3
I20260809 15:19:19.759142 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.758651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50348)
0809 15:19:19.758852 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.758961 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 15:19:19.760281 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.759815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50350)
0809 15:19:19.760021 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.760109 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260809 15:19:19.760545 8001 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.11.188:36695: Network error: Client connection negotiation failed: client connection to 127.15.11.188:36695: connect: Connection refused (error 111)
W20260809 15:19:19.763439 8090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:19:19.763513 8090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:19:19.763569 8090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:19:19.763618 8090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:19:19.763679 8090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:19:19.773356 8090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.11.148:34329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.11.190:42381,127.15.11.189:42337,127.15.11.188:36695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/krb5kdc/kudu_127.15.11.188.keytab
--principal=kudu/127.15.11.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.11.188:36695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.11.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 14:07:21 UTC on ccycloud.quasar-pgucrk.root.comops.site
build id 4576546
ASAN enabled
I20260809 15:19:19.774513 8090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:19:19.775717 8090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:19:19.787524 8095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:19.787439 8096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:19:19.787571 8098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:19:19.788174 8090 server_base.cc:1034] running on GCE node
I20260809 15:19:19.788867 8090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:19:19.790107 8090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:19:19.791357 8090 hybrid_clock.cc:648] HybridClock initialized: now 1786288759791280 us; error 49 us; skew 500 ppm
I20260809 15:19:19.791678 8090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:19:19.794943 8090 init.cc:377] Logged in from keytab as kudu/127.15.11.188@KRBTEST.COM (short username kudu)
I20260809 15:19:19.796504 8090 webserver.cc:460] Webserver started at http://127.15.11.188:38131/ using document root <none> and password file <none>
I20260809 15:19:19.797164 8090 fs_manager.cc:359] Metadata directory not provided
I20260809 15:19:19.797314 8090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:19:19.797645 8090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:19:19.800525 8090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/data/instance:
uuid: "80ee4ba0a7134eb78ff6972b36e1fe04"
format_stamp: "Formatted at 2026-08-09 15:19:19 on dist-test-slave-8w9v"
server_key: "c9b06b811c3ab3a11733fa8bef80118b"
server_key_iv: "eac693089f316b034ad0d6a2f74ff125"
server_key_version: "encryptionkey@0"
I20260809 15:19:19.801296 8090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/wal/instance:
uuid: "80ee4ba0a7134eb78ff6972b36e1fe04"
format_stamp: "Formatted at 2026-08-09 15:19:19 on dist-test-slave-8w9v"
server_key: "c9b06b811c3ab3a11733fa8bef80118b"
server_key_iv: "eac693089f316b034ad0d6a2f74ff125"
server_key_version: "encryptionkey@0"
I20260809 15:19:19.806334 8090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:19:19.809854 8106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:19:19.811206 8090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:19:19.811383 8090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/data,/tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/wal
uuid: "80ee4ba0a7134eb78ff6972b36e1fe04"
format_stamp: "Formatted at 2026-08-09 15:19:19 on dist-test-slave-8w9v"
server_key: "c9b06b811c3ab3a11733fa8bef80118b"
server_key_iv: "eac693089f316b034ad0d6a2f74ff125"
server_key_version: "encryptionkey@0"
I20260809 15:19:19.811596 8090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:19:19.840260 8090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:19:19.844511 8090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:19:19.844772 8090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:19:19.864405 8090 rpc_server.cc:307] RPC server started. Bound to: 127.15.11.188:36695
I20260809 15:19:19.864426 8163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.11.188:36695 every 8 connection(s)
I20260809 15:19:19.865835 8090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/data/info.pb
I20260809 15:19:19.869737 8164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:19:19.871950 15406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskFizLz9/build/asan/bin/kudu as pid 8090
I20260809 15:19:19.872192 15406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskFizLz9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786288392170773-15406-0/minicluster-data/master-2/wal/instance
I20260809 15:19:19.872737 15406 external_mini_cluster.cc:1428] Setting key e39a41ab3610998b3d19d0a1c5aa3ba1
I20260809 15:19:19.877712 8164 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:19:19.894845 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.880751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57396 (local address 127.15.11.190:42381)
0809 15:19:19.882007 (+ 1256us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.882013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.882032 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:19:19.882093 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.882096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.882173 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.882292 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.883461 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.884163 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.885614 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.886019 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.886155 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.889427 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.889432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.889435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.889471 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.891895 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.892737 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.892742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.892745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.892885 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.893252 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.893257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.893260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.893583 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.893808 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.894257 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.894456 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1113,"thread_start_us":983,"threads_started":1}
I20260809 15:19:19.894948 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.880689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.190:42381 (local address 127.0.0.1:57396)
0809 15:19:19.881526 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.881581 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:19.881748 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:19.882378 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:19.882385 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:19.883036 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:19.883305 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.883314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.884324 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.884328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.885457 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.885469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.886249 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.886256 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.886404 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.886976 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:19.886995 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:19.889229 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:19.892032 (+ 2803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.892041 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.892048 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.892615 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.892998 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.893006 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.893009 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.893147 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.893729 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:19.893738 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:19.893939 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 15:19:19.894147 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 15:19:19.894450 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":335,"threads_started":1}
I20260809 15:19:19.894944 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.874680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.15.11.188:36695)
0809 15:19:19.875552 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.875563 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.875607 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:19:19.877301 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.877323 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.877715 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.878026 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.879808 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.881880 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.883190 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.883825 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.884008 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.887962 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.887969 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.887980 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.888013 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.891037 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.891674 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.891682 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.891689 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.891843 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.892243 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.892248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.892253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.892822 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.893131 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.893574 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.893960 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":360,"threads_started":1}
I20260809 15:19:19.897274 8164 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.190@KRBTEST.COM
I20260809 15:19:19.910298 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.897934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.189:42337 (local address 127.0.0.1:50940)
0809 15:19:19.898125 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.898142 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:19.898222 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:19.898563 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:19.898566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:19.898912 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:19.899033 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.899040 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.899970 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.899976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.900865 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.900876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.901548 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.901553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.901729 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.902247 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:19.902269 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:19.904554 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:19.907833 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.907839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.907843 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.908314 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.908693 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.908697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.908701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.908843 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.909347 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:19.909351 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:19.909505 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 15:19:19.909638 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 15:19:19.909908 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260809 15:19:19.910339 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.898036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.15.11.189:42337)
0809 15:19:19.898212 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.898217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.898235 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:19:19.898352 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.898358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.898424 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.898542 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.899176 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.899840 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.900999 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.901404 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.901529 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.904747 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.904755 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.904762 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.904806 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.907704 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.908433 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.908439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.908442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.908579 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.908965 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.908970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.908973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.909227 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.909392 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.909742 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.909956 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:19:19.911185 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.898037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57412 (local address 127.15.11.190:42381)
0809 15:19:19.898197 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.898202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.898216 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:19:19.898317 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.898321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.898400 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.898517 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.899638 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.900333 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.901490 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.901916 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.902048 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.906205 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.906210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.906213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.906240 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.908702 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.909298 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.909304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.909308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.909426 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.909798 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.909803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.909806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.910065 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.910369 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.910669 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.910904 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:19:19.911690 8164 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.188@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.11.189@KRBTEST.COM
I20260809 15:19:19.925293 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.912569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50372)
0809 15:19:19.912701 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.912718 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:19.912818 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:19.913116 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:19.913122 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:19.913540 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:19.913659 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.913667 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.914933 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.914938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.916217 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.916229 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.917080 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.917084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.917249 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.917730 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:19.917752 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:19.919738 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:19.922903 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.922907 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.922911 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.923248 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.923595 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.923602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.923607 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.923748 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.924280 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:19.924285 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:19.924427 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 15:19:19.924572 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 15:19:19.924876 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":5}
I20260809 15:19:19.925315 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.912458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50372 (local address 127.15.11.188:36695)
0809 15:19:19.912611 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.912618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.912770 (+ 152us) server_negotiation.cc:407] Connection header received
0809 15:19:19.912928 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.912933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.912995 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.913080 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.913800 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.914797 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.916341 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.916947 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.917076 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.919947 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.919955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.919962 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.920004 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.922732 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.923370 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.923375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.923379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.923492 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.923852 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.923858 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.923861 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.924159 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.924317 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.924652 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.925024 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":175,"server-negotiator.queue_time_us":49}
I20260809 15:19:19.927793 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.912945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50944 (local address 127.15.11.189:42337)
0809 15:19:19.913085 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.913090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.913105 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:19:19.913147 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.913151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.913218 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.913312 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.914669 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.915715 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.917371 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.917981 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.918104 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.922450 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.922456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.922460 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.922497 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.925071 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.925845 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.925852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.925856 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.925997 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.926384 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.926390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.926393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.926704 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.926979 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.927243 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.927466 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 15:19:19.930962 8164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04: Bootstrap starting.
I20260809 15:19:19.932576 15406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:19:19.934885 8164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:19.936072 8164 log.cc:824] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:19.937763 8001 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } attempt: 4
I20260809 15:19:19.935660 7974 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57412
I20260809 15:19:19.939183 8164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04: No bootstrap required, opened a new log
Aug 09 15:19:19 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288758, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.190@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
I20260809 15:19:19.949851 8164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } }
I20260809 15:19:19.950266 8164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80ee4ba0a7134eb78ff6972b36e1fe04, State: Initialized, Role: FOLLOWER
I20260809 15:19:19.950796 8164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } }
I20260809 15:19:19.951226 8004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.938352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50388)
0809 15:19:19.938583 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:19.938601 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:19.938706 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:19.939161 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:19.939167 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:19.939678 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:19.939895 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.939909 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.940945 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.940952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.941860 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:19.941872 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.942657 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.942663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:19.942861 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.943464 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:19.943481 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:19.945512 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:19.948760 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.948766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.948770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.949124 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.949502 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:19.949507 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:19.949511 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:19.949659 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:19.950220 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:19.950225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:19.950404 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 15:19:19.950554 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 15:19:19.950818 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260809 15:19:19.951227 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:19.938421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50388 (local address 127.15.11.188:36695)
0809 15:19:19.938908 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:19.938912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:19.938925 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:19:19.938961 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:19.938964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:19.939013 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:19.939115 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:19.940048 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.940799 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.941999 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:19.942471 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:19.942633 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:19.945692 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:19.945696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:19.945699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:19.945727 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:19.948630 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.949248 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.949256 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.949263 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.949390 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:19.949769 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:19.949774 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:19.949778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:19.950092 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:19.950263 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:19.950666 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 15:19:19.950876 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301}
I20260809 15:19:19.952596 8171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:19.953037 8171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:19.953713 8164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:19.956980 8001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad: Bootstrap starting.
I20260809 15:19:19.960693 8001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:19.961935 8001 log.cc:824] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad: Log is configured to *not* fsync() on all Append() calls
I20260809 15:19:19.964655 8001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad: No bootstrap required, opened a new log
I20260809 15:19:19.969794 8001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } }
I20260809 15:19:19.970212 8001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f93b17fa5f34ef6bf3a30074508b2ad, State: Initialized, Role: FOLLOWER
I20260809 15:19:19.970674 8001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } }
I20260809 15:19:19.972918 8001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:19.972957 7974 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:19:19.973147 7974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:19:19.974963 8183 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:19:19.975106 8183 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:19:19.976404 7974 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } }
I20260809 15:19:19.977829 7974 leader_election.cc:288] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [CANDIDATE]: Term 1 election: Requested vote from peers d6dd7b1d50d84c06b93c8a0c21cbe79d (127.15.11.189:42337), 80ee4ba0a7134eb78ff6972b36e1fe04 (127.15.11.188:36695)
I20260809 15:19:19.979251 8137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04"
I20260809 15:19:19.979732 8137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:19.971597 8184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:19.983387 8137 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f93b17fa5f34ef6bf3a30074508b2ad in term 1.
I20260809 15:19:19.983415 8184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:19.984066 7948 leader_election.cc:302] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80ee4ba0a7134eb78ff6972b36e1fe04, 8f93b17fa5f34ef6bf3a30074508b2ad; no voters:
I20260809 15:19:19.985699 8184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:19:19.971731 8090 master_runner.cc:186] Error getting master registration for 127.15.11.190:42381: OK, instance_id { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" instance_seqno: 1786288758801199 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:19:19.990461 8090 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:19:19.990576 8184 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [term 1 LEADER]: Becoming Leader. State: Replica: 8f93b17fa5f34ef6bf3a30074508b2ad, State: Running, Role: LEADER
I20260809 15:19:19.991217 8184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } }
I20260809 15:19:19.999145 8190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f93b17fa5f34ef6bf3a30074508b2ad. Latest consensus state: current_term: 1 leader_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:19.999569 8190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:20.000514 8197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:19:20.025573 8197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:19:20.034773 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.011313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50394 (local address 127.15.11.188:36695)
0809 15:19:20.011514 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.011520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.017606 (+ 6086us) server_negotiation.cc:407] Connection header received
0809 15:19:20.017759 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.017782 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.017897 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.017987 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.020169 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.021279 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.023131 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.023810 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.023902 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.027659 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:20.027665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:20.027670 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:20.027709 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:20.031460 (+ 3751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:20.032284 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:20.032291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:20.032298 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:20.032473 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:20.032957 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:20.032962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:20.032965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:20.033306 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:20.033560 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.034117 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.034385 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260809 15:19:20.040239 7951 tablet.cc:2369] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:19:20.048486 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.007929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50956 (local address 127.15.11.189:42337)
0809 15:19:20.008134 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.008161 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.013596 (+ 5435us) server_negotiation.cc:407] Connection header received
0809 15:19:20.017828 (+ 4232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.017835 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.017922 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.017988 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.032810 (+ 14822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.033921 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.037772 (+ 3851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.038468 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.038604 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.041651 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:20.041657 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:20.041663 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:20.041705 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:20.045180 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:20.046046 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:20.046052 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:20.046056 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:20.046216 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:20.046705 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:20.046712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:20.046716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:20.047074 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:20.047327 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.047848 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.048081 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:19:20.051486 8082 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } attempt: 5
I20260809 15:19:20.055254 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.005285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.15.11.190:42381)
0809 15:19:20.005469 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.005476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.007571 (+ 2095us) server_negotiation.cc:407] Connection header received
0809 15:19:20.008820 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.008829 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.008951 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.009056 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.025843 (+ 16787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.027074 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.035159 (+ 8085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.035889 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.036024 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.048476 (+ 12452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:20.048485 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:20.048488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:20.048532 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:20.051979 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:20.052813 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:20.052818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:20.052822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:20.052967 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:20.053424 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:20.053428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:20.053431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:20.053842 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:20.054107 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.054661 (+ 554us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.054879 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 15:19:20 dist-test-slave-8w9v krb5kdc[7914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786288759, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.11.189@KRBTEST.COM for kudu/127.15.11.188@KRBTEST.COM
I20260809 15:19:20.064621 8137 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [term 1 FOLLOWER]: Refusing update from remote peer 8f93b17fa5f34ef6bf3a30074508b2ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:19:20.065665 8190 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:19:20.079290 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.052339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.15.11.188:36695)
0809 15:19:20.052506 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.052511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.052529 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:19:20.052580 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.052586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.052658 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.052750 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.053841 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.054960 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.057012 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.057687 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.057835 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.071247 (+ 13412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:19:20.071263 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:19:20.071269 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:19:20.071311 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:19:20.075137 (+ 3826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:20.076077 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:20.076082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:20.076086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:20.076298 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:19:20.076763 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:19:20.076768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:19:20.076771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:19:20.077144 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:19:20.077378 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.078571 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.078891 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:19:20.079629 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.051979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.11.188:36695 (local address 127.0.0.1:50402)
0809 15:19:20.052199 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 15:19:20.052223 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 15:19:20.052350 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:19:20.052937 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:19:20.052942 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:19:20.053435 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:19:20.053641 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:20.053670 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.055204 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.055210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:20.056674 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:19:20.056689 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.058059 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.058064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:19:20.058319 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.058907 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:19:20.058929 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:19:20.071009 (+ 12080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:19:20.075336 (+ 4327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:20.075341 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:20.075345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:20.075900 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:20.076439 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:19:20.076446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:19:20.076451 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:19:20.076623 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:19:20.077448 (+ 825us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:19:20.077455 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:19:20.077620 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 15:19:20.077786 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 15:19:20.078085 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 15:19:20.086359 8082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d: Bootstrap starting.
I20260809 15:19:20.092723 8082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d: Neither blocks nor log segments found. Creating new log.
I20260809 15:19:20.095410 8082 log.cc:824] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d: Log is configured to *not* fsync() on all Append() calls
W20260809 15:19:20.096879 8112 tablet.cc:2369] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:19:20.105373 8171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f93b17fa5f34ef6bf3a30074508b2ad. Latest consensus state: current_term: 1 leader_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:20.105728 8171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:20.109226 8082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d: No bootstrap required, opened a new log
I20260809 15:19:20.115967 8184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:20.116385 8184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:20.099144 7928 master_runner.cc:186] Error getting master registration for 127.15.11.189:42337: OK, instance_id { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" instance_seqno: 1786288759324760 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:19:20.124047 7928 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:19:20.126230 8197 catalog_manager.cc:1347] Generated new cluster ID: a8270583037b46f48e0f20a5d920e326
I20260809 15:19:20.126345 8197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:19:20.128710 8171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:20.128932 8171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:20.136695 8082 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } }
I20260809 15:19:20.137351 8082 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6dd7b1d50d84c06b93c8a0c21cbe79d, State: Initialized, Role: FOLLOWER
I20260809 15:19:20.138146 8082 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } }
I20260809 15:19:20.148859 8055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d"
I20260809 15:19:20.149296 8054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:19:20.149513 8082 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:19:20.152755 8054 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [term 1 FOLLOWER]: Refusing update from remote peer 8f93b17fa5f34ef6bf3a30074508b2ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:19:20.153674 8190 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:19:20.156339 8055 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f93b17fa5f34ef6bf3a30074508b2ad in term 1.
I20260809 15:19:20.158296 8218 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:20.158653 8218 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:20.167510 8197 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:19:20.169270 8197 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:19:20.191226 8218 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f93b17fa5f34ef6bf3a30074508b2ad. Latest consensus state: current_term: 1 leader_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:20.191547 8218 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:20.192641 8184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:20.193011 8184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad [sys.catalog]: This master's current role is: LEADER
I20260809 15:19:20.193732 8226 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f93b17fa5f34ef6bf3a30074508b2ad" member_type: VOTER last_known_addr { host: "127.15.11.190" port: 42381 } } peers { permanent_uuid: "d6dd7b1d50d84c06b93c8a0c21cbe79d" member_type: VOTER last_known_addr { host: "127.15.11.189" port: 42337 } } peers { permanent_uuid: "80ee4ba0a7134eb78ff6972b36e1fe04" member_type: VOTER last_known_addr { host: "127.15.11.188" port: 36695 } } }
I20260809 15:19:20.194111 8226 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:19:20.200663 8197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f93b17fa5f34ef6bf3a30074508b2ad: Generated new TSK 0
I20260809 15:19:20.201682 8197 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:19:20.231302 8238 catalog_manager.cc:1259] Loaded cluster ID: a8270583037b46f48e0f20a5d920e326
I20260809 15:19:20.231406 8238 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d: loading cluster ID for follower catalog manager: success
I20260809 15:19:20.235131 8238 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d: acquiring CA information for follower catalog manager: success
I20260809 15:19:20.237563 8238 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d6dd7b1d50d84c06b93c8a0c21cbe79d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 15:19:20.357977 8169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.334629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.15.11.190:42381)
0809 15:19:20.334842 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.334848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.355708 (+ 20860us) server_negotiation.cc:407] Connection header received
0809 15:19:20.356309 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.356316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.356407 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.356487 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.357709 (+ 1222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57442: BlockingRecv error: recv got EOF from 127.0.0.1:57442 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 15:19:20.360400 8167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.340980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50418 (local address 127.15.11.188:36695)
0809 15:19:20.341154 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.341160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.358651 (+ 17491us) server_negotiation.cc:407] Connection header received
0809 15:19:20.358905 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.358912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.358992 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.359064 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.360104 (+ 1040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50418: BlockingRecv error: recv got EOF from 127.0.0.1:50418 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 15:19:20.363706 8087 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.361555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.15.11.189:42337)
0809 15:19:20.361791 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.361797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.362224 (+ 427us) server_negotiation.cc:407] Connection header received
0809 15:19:20.362449 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.362456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.362534 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.362608 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.363502 (+ 894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50962: BlockingRecv error: recv got EOF from 127.0.0.1:50962 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 15:19:20.416535 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.396755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50966 (local address 127.15.11.189:42337)
0809 15:19:20.396962 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.396968 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.408314 (+ 11346us) server_negotiation.cc:407] Connection header received
0809 15:19:20.408963 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.408972 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.409209 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.409343 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.410101 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.411258 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.413343 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.414010 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.414105 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.414916 (+ 811us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:20.415602 (+ 686us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:20.415720 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.416080 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.416201 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 15:19:20.476068 8167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.463017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.15.11.188:36695)
0809 15:19:20.463212 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.463218 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.473759 (+ 10541us) server_negotiation.cc:407] Connection header received
0809 15:19:20.474332 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.474339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.474508 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.474578 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.475778 (+ 1200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50428: BlockingRecv error: recv got EOF from 127.0.0.1:50428 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:19:20.530592 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.511733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.15.11.189:42337)
0809 15:19:20.511945 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.511951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.522682 (+ 10731us) server_negotiation.cc:407] Connection header received
0809 15:19:20.523232 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.523240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.523363 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.523434 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.524279 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.525471 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.527733 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.528472 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.528585 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.529667 (+ 1082us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:20.529837 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:20.529912 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.530176 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.530268 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 15:19:20.574084 8167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.564171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.15.11.188:36695)
0809 15:19:20.564406 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.564412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.571824 (+ 7412us) server_negotiation.cc:407] Connection header received
0809 15:19:20.572396 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.572403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.572508 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.572597 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.573809 (+ 1212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50434: BlockingRecv error: recv got EOF from 127.0.0.1:50434 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 15:19:20.623107 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.615737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.15.11.189:42337)
0809 15:19:20.616032 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.616038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.616650 (+ 612us) server_negotiation.cc:407] Connection header received
0809 15:19:20.616885 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.616892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.616992 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.617094 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.617640 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.618860 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.620853 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.621559 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.621686 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.622197 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:20.622357 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:20.622447 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.622643 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.622735 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
W20260809 15:19:20.636684 8167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.634121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50450 (local address 127.15.11.188:36695)
0809 15:19:20.634324 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.634330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.634941 (+ 611us) server_negotiation.cc:407] Connection header received
0809 15:19:20.635155 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.635163 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.635289 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.635413 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.636421 (+ 1008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50450: BlockingRecv error: recv got EOF from 127.0.0.1:50450 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:19:20.651041 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.644061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50986 (local address 127.15.11.189:42337)
0809 15:19:20.644319 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.644324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.644668 (+ 344us) server_negotiation.cc:407] Connection header received
0809 15:19:20.644840 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.644845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.644928 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.645024 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.645528 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.646793 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.648809 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.649516 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.649643 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.650199 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:20.650379 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:20.650471 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.650673 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.650757 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
W20260809 15:19:20.662448 8167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.658397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50456 (local address 127.15.11.188:36695)
0809 15:19:20.658682 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.658688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.660719 (+ 2031us) server_negotiation.cc:407] Connection header received
0809 15:19:20.660989 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.660996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.661104 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.661215 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.662224 (+ 1009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50456: BlockingRecv error: recv got EOF from 127.0.0.1:50456 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
I20260809 15:19:20.681926 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.674616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.15.11.189:42337)
0809 15:19:20.674807 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.674813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.675402 (+ 589us) server_negotiation.cc:407] Connection header received
0809 15:19:20.675573 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.675580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.675676 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.675785 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.676361 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.677521 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.679471 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.680286 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.680417 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.680923 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:20.681071 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:20.681168 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.681408 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.681495 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 15:19:20.691804 8167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.689491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.15.11.188:36695)
0809 15:19:20.689677 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.689683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.690216 (+ 533us) server_negotiation.cc:407] Connection header received
0809 15:19:20.690362 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.690368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.690490 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.690604 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.691531 (+ 927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50462: BlockingRecv error: recv got EOF from 127.0.0.1:50462 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:19:20.718400 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.711103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51006 (local address 127.15.11.189:42337)
0809 15:19:20.711275 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.711282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.711861 (+ 579us) server_negotiation.cc:407] Connection header received
0809 15:19:20.712075 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.712081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.712233 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.712354 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.712915 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.714194 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.716278 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.716972 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.717102 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.717647 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:20.717787 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:20.717878 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.718054 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.718142 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 15:19:20.728058 8167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.725844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50474 (local address 127.15.11.188:36695)
0809 15:19:20.726022 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.726028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.726552 (+ 524us) server_negotiation.cc:407] Connection header received
0809 15:19:20.726695 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.726701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.726822 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.726939 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.727819 (+ 880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50474: BlockingRecv error: recv got EOF from 127.0.0.1:50474 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:19:20.768631 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.763269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.15.11.189:42337)
0809 15:19:20.763549 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.763555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.763850 (+ 295us) server_negotiation.cc:407] Connection header received
0809 15:19:20.764091 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.764095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.764230 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.764332 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.764794 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.765499 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.766795 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.767258 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.767401 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.767824 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:20.767943 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:20.768048 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.768241 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.768320 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 15:19:20.776564 8167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.774509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50486 (local address 127.15.11.188:36695)
0809 15:19:20.774706 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.774712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.775169 (+ 457us) server_negotiation.cc:407] Connection header received
0809 15:19:20.775325 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.775331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.775439 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.775543 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.776352 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50486: BlockingRecv error: recv got EOF from 127.0.0.1:50486 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:19:20.848474 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.843768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51034 (local address 127.15.11.189:42337)
0809 15:19:20.843955 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.843959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.844384 (+ 425us) server_negotiation.cc:407] Connection header received
0809 15:19:20.844570 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.844574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.844644 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.844728 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.845118 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.845838 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.847071 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.847433 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.847517 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.847863 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:20.847954 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:20.848034 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.848193 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.848252 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 15:19:20.855834 8167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:19:20.854110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50494 (local address 127.15.11.188:36695)
0809 15:19:20.854331 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.854337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.854600 (+ 263us) server_negotiation.cc:407] Connection header received
0809 15:19:20.854733 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.854737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.854819 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.854901 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:19:20.855637 (+ 736us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50494: BlockingRecv error: recv got EOF from 127.0.0.1:50494 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 15:19:20.977190 8183 catalog_manager.cc:1259] Loaded cluster ID: a8270583037b46f48e0f20a5d920e326
I20260809 15:19:20.977330 8183 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04: loading cluster ID for follower catalog manager: success
I20260809 15:19:20.979784 8183 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04: acquiring CA information for follower catalog manager: success
I20260809 15:19:20.981734 8183 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 80ee4ba0a7134eb78ff6972b36e1fe04: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:19:20.991873 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.987321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.15.11.189:42337)
0809 15:19:20.987504 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.987509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.987851 (+ 342us) server_negotiation.cc:407] Connection header received
0809 15:19:20.988029 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.988034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.988103 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.988250 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.988609 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.989277 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.990507 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.990898 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:20.990983 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:20.991336 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:20.991433 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:20.991494 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:20.991619 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 15:19:20.991677 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:19:21.003264 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:20.997664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50496 (local address 127.15.11.188:36695)
0809 15:19:20.997883 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:20.997889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:20.998228 (+ 339us) server_negotiation.cc:407] Connection header received
0809 15:19:20.998451 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:20.998458 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:20.998634 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:20.998747 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:20.999231 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:20.999859 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:21.001194 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:21.001685 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:21.001774 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:21.002180 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:21.002711 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:21.002815 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:21.002883 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 15:19:21.002946 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 15:19:21.016568 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:21.008965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50504 (local address 127.15.11.188:36695)
0809 15:19:21.009166 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:21.009175 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:21.010431 (+ 1256us) server_negotiation.cc:407] Connection header received
0809 15:19:21.010612 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:21.010618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:21.010710 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:21.010799 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:21.011483 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:21.012474 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:21.014377 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:21.015021 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:21.015127 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:21.015763 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:21.015900 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:21.016012 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:21.016218 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 15:19:21.016293 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:19:21.017230 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:21.009805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51058 (local address 127.15.11.189:42337)
0809 15:19:21.010008 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:21.010015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:21.011119 (+ 1104us) server_negotiation.cc:407] Connection header received
0809 15:19:21.011287 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:21.011295 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:21.011392 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:21.011479 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:21.012012 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:21.013159 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:21.015084 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:21.015831 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:21.015981 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:21.016527 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:21.016661 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:21.016728 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:21.016952 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 15:19:21.017023 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:19:21.018119 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:19:21.008785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.15.11.190:42381)
0809 15:19:21.011586 (+ 2801us) server_negotiation.cc:206] Beginning negotiation
0809 15:19:21.011592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:19:21.011608 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:19:21.011715 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:19:21.011721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:19:21.011933 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:19:21.012096 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:19:21.012714 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:21.013597 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:21.015500 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:19:21.016176 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:19:21.016765 (+ 589us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:19:21.016963 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:19:21.017412 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:19:21.017575 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 15:19:21.017736 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 15:19:21.017825 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2672,"thread_start_us":2576,"threads_started":1}
I20260809 15:19:21.022516 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 7928
I20260809 15:19:21.024067 7996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:19:21.132844 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 7928
I20260809 15:19:21.158237 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 8009
I20260809 15:19:21.159363 8077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:19:21.268851 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 8009
I20260809 15:19:21.292744 15406 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskFizLz9/build/asan/bin/kudu pid 8090
I20260809 15:19:21.293944 8159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 15:19:21.400245 15406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskFizLz9/build/asan/bin/kudu with pid 8090
2026-08-09T15:19:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4517 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9548 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (369200 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST